-- Logs begin at Thu 2022-06-30 14:48:37 UTC, end at Thu 2022-06-30 16:09:08 UTC. -- Jun 30 15:36:29.249264 dv-cinder-797608-40-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 15:36:29.256722 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 15:36:29 2022] *** Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: nodename: dv-cinder-797608-40-iscsi Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: machine: x86_64 Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: clock source: unix Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: pcre jit disabled Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: detected number of CPU cores: 4 Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: current working directory: / Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: detected binary path: /usr/bin/uwsgi-core Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: your processes number limit is 47680 Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: your memory page size is 4096 bytes Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: detected max file descriptor number: 2048 Jun 30 15:36:29.262247 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: lock engine: pthread robust mutexes Jun 30 15:36:29.262966 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: thunder lock: enabled Jun 30 15:36:29.262966 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 15:36:29.262966 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 15:36:29.325752 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: Python main interpreter initialized at 0x559a686679c0 Jun 30 15:36:29.325752 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: python threads support enabled Jun 30 15:36:29.325752 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: your server socket listen backlog is limited to 100 connections Jun 30 15:36:29.325752 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: your mercy for graceful operations on workers is 90 seconds Jun 30 15:36:29.326142 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 15:36:29.326214 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: *** Operational MODE: preforking *** Jun 30 15:36:29.326476 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: *** uWSGI is running in multiple interpreter mode *** Jun 30 15:36:29.326476 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: spawned uWSGI master process (pid: 91944) Jun 30 15:36:29.326732 dv-cinder-797608-40-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 15:36:29.327322 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: spawned uWSGI worker 1 (pid: 91957, cores: 1) Jun 30 15:36:29.327654 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: spawned uWSGI worker 2 (pid: 91958, cores: 1) Jun 30 15:36:29.327820 dv-cinder-797608-40-iscsi devstack@keystone.service[91944]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 15:36:34.732130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 15:36:34.732130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: from cryptography.utils import int_from_bytes Jun 30 15:36:34.732952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 15:36:34.732952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: from cryptography.utils import int_from_bytes Jun 30 15:36:34.733441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 15:36:34.733441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: from cryptography.utils import int_from_bytes Jun 30 15:36:34.733958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 15:36:34.733958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: from cryptography.utils import int_from_bytes Jun 30 15:36:34.781285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 15:36:34.781493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 15:36:34.781647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] command line args: [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 15:36:34.781909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 15:36:34.782041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ================================================================================ {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 15:36:34.782323 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] admin_token = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.782563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] config_dir = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.782728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.782946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 15:36:34.783079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] config_source = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.783164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 15:36:34.783234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] debug = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.783307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] command line args: [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 15:36:34.783373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.783610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 15:36:34.783689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.783751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ================================================================================ {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 15:36:34.783818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.783879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] admin_token = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] config_dir = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] config_source = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] list_limit = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] debug = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] log_config_append = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.784914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] log_dir = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] log_file = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.785909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] list_limit = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] log_config_append = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] log_dir = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] log_file = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.786975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.787924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.788946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789058 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] publish_errors = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789817 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.789951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] standard_threads = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] publish_errors = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.790953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] use_journal = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] use_json = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] use_stderr = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] use_syslog = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.791894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.792024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] standard_threads = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.792133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.792273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.792413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.792492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.792668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.792757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.792842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] use_journal = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.792916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.792999 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] use_json = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.793134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.793226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] use_stderr = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.793307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] use_syslog = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.793385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.793521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 15:36:34.793601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.793771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.793858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.794958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.795977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.796976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.797104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.797218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.797462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.797560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.797714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.797794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.797961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.798982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.799964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.800910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.801963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.802960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.803111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.803197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.803336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.803414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.803708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.803835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.803947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804803 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.804996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.805938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.806926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.807874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.808936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.809863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.810957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.811981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.812160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.812251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.812397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.812476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.812627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.812706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.812863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.813969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.814947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.815982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] auth.external = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.816993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] auth.password = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] auth.token = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.817982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] auth.external = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.818888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] auth.password = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] auth.token = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] credential.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819809 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.819957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.820922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] credential.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.821992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.822188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.822268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.822435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.822523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.822693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.822784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.822931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.823916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.824955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.825122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.825203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.825357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.825498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.825593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.825768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.825849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.826986 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.827909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.828945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.829985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830761 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.830995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.831912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832803 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.832891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.833881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.834965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835572 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.835915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.836903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.837852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.838938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.839946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.840952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.841859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.842940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.843927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.844942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.845981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.846907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.847842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848707 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.848936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] role.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] role.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.849944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.850869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.851944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.852900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] role.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] role.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.853862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.854989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.855911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.856886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.857842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.858943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.859882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.860862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.connection = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.861940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.862968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.863901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864074 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.864997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.connection = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.865941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.866031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.866108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91957) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 15:36:34.866213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.866443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.866677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.866905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.867136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.867364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.867592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.867829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.868083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.868324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.868553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.868792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.869021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.869257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.869481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.869716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.869944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 15:36:34.870109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91958) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 15:36:34.896771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.897035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.897230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.900417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.900609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.900863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.905767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.909448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.933723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.933941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.935960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:34.936214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.081375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.082559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.130244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.130428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.131727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.131945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.194427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.194599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.194823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.194906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.195029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.195121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.195251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.195383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.195455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.195534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.195759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.196027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.219461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.219616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.219856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.220092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.220322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.220424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.220524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.220707 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.220860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.220957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.221169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.221359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.221604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.221850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.231876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.233069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.244610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.244855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.245122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.245309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.245411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.245542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.245669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.245808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.245943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.246078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.246210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91957) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.246388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.246388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.246388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91958) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 15:36:35.265984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.265984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 15:36:35.266369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.266369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 15:36:35.266673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.266673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 15:36:35.266967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.266967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 15:36:35.267237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.267377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.267528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.267670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.267811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.268487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.268621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.268775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.276100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.276376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.276502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.276654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.276787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.276977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.277068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.277311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.285256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.286364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.288831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.289111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.289393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.289673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.289800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.290020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.290117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.290283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.290409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.290548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.290691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.290821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.290947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.291077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.291208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.291336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.291474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.291603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.291732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.291858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.292060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.292370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.323869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.324269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.325534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.325822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.326023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.326257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.331119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.331474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.331653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.331830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.331939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.332397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.339187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.339472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.340035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.340414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.344005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.344301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.344577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.344859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.345124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.345339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.345469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.345614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.345743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.345890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.346033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.346133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.346431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.346681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.346941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.347199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.364722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.364991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.365972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.366242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.369587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.370904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.373254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.373545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.375474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.375757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.378675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.380920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.382277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.384840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.385939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.386290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.386899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.387174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.387460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.387739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.389030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.389323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.389597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.389869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.390142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.390416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.406843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.407121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.407394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.407615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.409958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.410276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.410547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.410869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.415550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.417293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.417577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.419130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.420778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.421017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.421153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.421361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.421491 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.421632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.421847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.424887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.425183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.425503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.425815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.426150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.435255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.435531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.435813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.436111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.436396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.438066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.438364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.440863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.441138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.441416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.441688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.441963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.442228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.442504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.456332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.459587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.461709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.462940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.464746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.465097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.466366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.466629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.466918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.468413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.470149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.471885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.472188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.472470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.474343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.475949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.479194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.479491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.479840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.479968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.480306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.481597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.485307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.485597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.485874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.486154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.489453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.489738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.490021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.495374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.495687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.495966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.497945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91957) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.498223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.498502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.498771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.499038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.499401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.499688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.499923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.500258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.500609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.500816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.501082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.501340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.501607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91957) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.503970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91958) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 15:36:35.504265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.504539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.504806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.505073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.505336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.505615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.505875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.506247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.506512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.506781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.507040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.507287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.507548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91958) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 15:36:35.539933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559a686679c0 pid: 91957 (default app) Jun 30 15:36:35.543373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f362f-c83b-48e3-bb08-808e5dfcadc9 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:35.543658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f362f-c83b-48e3-bb08-808e5dfcadc9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:35.543931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f362f-c83b-48e3-bb08-808e5dfcadc9 None None] PATH_INFO: `/v3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:35.547130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559a686679c0 pid: 91958 (default app) Jun 30 15:36:35.548242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Thu Jun 30 15:36:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:43.239251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8856b89-b40c-4ed1-a456-7f647debbce5 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:43.239877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8856b89-b40c-4ed1-a456-7f647debbce5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:43.239877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8856b89-b40c-4ed1-a456-7f647debbce5 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:43.244164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:36:43.250590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:43.251341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:43.251466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:43.324854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: INFO dbcounter [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] Registered counter for database keystone Jun 30 15:36:43.349605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG oslo_db.sqlalchemy.engines [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91957) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 15:36:43.360142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writer thread running {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 15:36:43.554000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG passlib.handlers.bcrypt [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91957) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 15:36:43.554345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG passlib.handlers.bcrypt [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91957) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 15:36:43.616864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:43.618507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:43.756639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 15:36:43.987621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8856b89-b40c-4ed1-a456-7f647debbce5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:44.066588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: INFO dbcounter [None req-b8856b89-b40c-4ed1-a456-7f647debbce5 None None] Registered counter for database keystone Jun 30 15:36:44.088453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG oslo_db.sqlalchemy.engines [None req-b8856b89-b40c-4ed1-a456-7f647debbce5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91958) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 15:36:44.100637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writer thread running {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 15:36:44.222055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8856b89-b40c-4ed1-a456-7f647debbce5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:44.227512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400754-c461-43c2-8458-086465ec9a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:44.228212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400754-c461-43c2-8458-086465ec9a4e admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:44.228499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400754-c461-43c2-8458-086465ec9a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:44.228772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400754-c461-43c2-8458-086465ec9a4e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:44.238256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1400754-c461-43c2-8458-086465ec9a4e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:44.290011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c1400754-c461-43c2-8458-086465ec9a4e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 15:36:44.290543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1400754-c461-43c2-8458-086465ec9a4e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:44.300879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Thu Jun 30 15:36:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 15:36:44.310258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:44.328770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ec7dc-ca6d-49ae-ae73-04a49c16bf35 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:44.335105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c1bac-8d9f-4ac9-94ec-0effefde717c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:44.336266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c1bac-8d9f-4ac9-94ec-0effefde717c admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:44.336552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c1bac-8d9f-4ac9-94ec-0effefde717c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:44.336824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c1bac-8d9f-4ac9-94ec-0effefde717c admin admin] PATH_INFO: `/v3/projects/052c2602e738443cb84400121f999c4f` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:44.338407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c1bac-8d9f-4ac9-94ec-0effefde717c admin admin] RBAC: Authorizing `identity:get_project(project_id=052c2602e738443cb84400121f999c4f)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:44.398948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3e0c1bac-8d9f-4ac9-94ec-0effefde717c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 15:36:44.399294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c1bac-8d9f-4ac9-94ec-0effefde717c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:44.404025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Thu Jun 30 15:36:44 2022] GET /identity/v3/projects/052c2602e738443cb84400121f999c4f => generated 344 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:46.389298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6517-568a-4119-ba4a-89f8a08ca82a None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:46.390173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6517-568a-4119-ba4a-89f8a08ca82a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:46.390173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d6517-568a-4119-ba4a-89f8a08ca82a None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:46.397026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:46.402640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d6113-8864-4a20-99e5-cd7df7eb1e3e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:46.403065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d6113-8864-4a20-99e5-cd7df7eb1e3e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:46.403152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d6113-8864-4a20-99e5-cd7df7eb1e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:46.425893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-290d6113-8864-4a20-99e5-cd7df7eb1e3e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:46.427428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-290d6113-8864-4a20-99e5-cd7df7eb1e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:46.437506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 30 15:36:46.460607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d6517-568a-4119-ba4a-89f8a08ca82a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:46.481998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d6517-568a-4119-ba4a-89f8a08ca82a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:46.487361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b642ac-7c60-41e6-a3ce-c183da12ea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:46.488022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b642ac-7c60-41e6-a3ce-c183da12ea3c admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:46.488327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b642ac-7c60-41e6-a3ce-c183da12ea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:46.488603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b642ac-7c60-41e6-a3ce-c183da12ea3c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:46.496616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b642ac-7c60-41e6-a3ce-c183da12ea3c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:46.498418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b642ac-7c60-41e6-a3ce-c183da12ea3c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:46.507450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Thu Jun 30 15:36:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:36:46.513727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d6113-8864-4a20-99e5-cd7df7eb1e3e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:46.523672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d6113-8864-4a20-99e5-cd7df7eb1e3e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:46.528993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fc0a0-6af5-465d-8e4a-6703f6534094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:46.529863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fc0a0-6af5-465d-8e4a-6703f6534094 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:46.530146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fc0a0-6af5-465d-8e4a-6703f6534094 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:46.530419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fc0a0-6af5-465d-8e4a-6703f6534094 admin admin] PATH_INFO: `/v3/users/3a26439857af474a895fe8556a7e3453` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:46.531958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fc0a0-6af5-465d-8e4a-6703f6534094 admin admin] RBAC: Authorizing `identity:get_user(user_id=3a26439857af474a895fe8556a7e3453)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:46.533826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fc0a0-6af5-465d-8e4a-6703f6534094 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:46.535414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Thu Jun 30 15:36:46 2022] GET /identity/v3/users/3a26439857af474a895fe8556a7e3453 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:36:48.596442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ed18-1f26-47c2-8f1b-7e996dd24a0f None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:48.596976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ed18-1f26-47c2-8f1b-7e996dd24a0f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:48.596976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0ed18-1f26-47c2-8f1b-7e996dd24a0f None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:48.598225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:48.603805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a863-e3cf-466b-90c8-c3b54892aab1 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:48.604280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a863-e3cf-466b-90c8-c3b54892aab1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:48.604368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9a863-e3cf-466b-90c8-c3b54892aab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:48.615407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3ecd7-1d3f-47e8-8434-48743da20827 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:48.615688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3ecd7-1d3f-47e8-8434-48743da20827 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:48.615957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3ecd7-1d3f-47e8-8434-48743da20827 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:48.617327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:48.621624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-67d9a863-e3cf-466b-90c8-c3b54892aab1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:48.622667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:48.622827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-67d9a863-e3cf-466b-90c8-c3b54892aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:48.622987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:48.623230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:48.663843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG passlib.handlers.bcrypt [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91958) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 15:36:48.664400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG passlib.handlers.bcrypt [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91958) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 15:36:48.704774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:36:48.709775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:48.711399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:48.727927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a863-e3cf-466b-90c8-c3b54892aab1 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:48.752794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:48.813086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:36:48.833615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9a863-e3cf-466b-90c8-c3b54892aab1 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:48.835170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:48.868326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:48.869069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:48.869353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:48.869630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:48.881163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:36:48.892909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:36:48.893881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:36:48.894731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:36:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:36:48.900150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:48.939982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b70532-26e3-4cc9-9a73-3c96701bf52e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:48.943094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:48.979448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:48.980483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:48.980762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:48.981009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:48.981186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:49.003432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293f5d8-bd56-48e4-af66-aa588e4321d4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:49.037382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:49.038055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:49.038338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:49.038618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:49.039364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:36:49.039673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:49.041153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:49.047799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:49.049348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:49.053191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 8/15] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:36:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:49.058815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:49.146363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 8/16] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:36:49.152017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:49.155622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:49.156555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab9a21-8dbb-4679-8b9c-a10ca3125483 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:49.189254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:49.190140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:49.190418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:49.190689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] PATH_INFO: `/v3/users/3a26439857af474a895fe8556a7e3453` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:49.191910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=3a26439857af474a895fe8556a7e3453)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:49.193442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:49.195041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 9/17] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:36:49 2022] GET /identity/v3/users/3a26439857af474a895fe8556a7e3453 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:49.200516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:49.215358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:49.251093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f03b0-09d2-4e3c-9b51-a6efffeae4aa None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:49.289905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:49.290714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:49.290992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:49.291259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:49.303558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65924b-cc0b-4872-b888-95051d484cb7 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:49.310229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:49.311739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:36:49.334156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:36:49.335021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:36:49.335840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:36:49.335840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:36:49.335840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:36:49.335840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:36:49.335840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 9/18] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:36:49.336589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:49.337427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:49.337706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:49.337980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:49.339209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:49.340701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:49.341244 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:49.342307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 10/19] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:36:49.347931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:49.398461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:49.430305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:49.448280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3dc96f-452f-4b8d-b1fe-f774da3871fb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:49.452356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf110c7-2bd1-4366-a2e7-52b8865b8432 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:49.486007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdcf60-2b99-4f3e-acac-467ff94a9d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:49.486727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdcf60-2b99-4f3e-acac-467ff94a9d06 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:49.487049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdcf60-2b99-4f3e-acac-467ff94a9d06 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:49.487360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdcf60-2b99-4f3e-acac-467ff94a9d06 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:49.488167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdcf60-2b99-4f3e-acac-467ff94a9d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:36:49.488500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdcf60-2b99-4f3e-acac-467ff94a9d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:49.488708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2adc0a-6501-4d66-9611-9239ab8af839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:49.489366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2adc0a-6501-4d66-9611-9239ab8af839 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:49.489647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2adc0a-6501-4d66-9611-9239ab8af839 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:49.489921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2adc0a-6501-4d66-9611-9239ab8af839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:49.490016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdcf60-2b99-4f3e-acac-467ff94a9d06 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:49.490748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2adc0a-6501-4d66-9611-9239ab8af839 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.domain.id=default, user.id=3a26439857af474a895fe8556a7e3453) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:36:49.491055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2adc0a-6501-4d66-9611-9239ab8af839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:49.492530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2adc0a-6501-4d66-9611-9239ab8af839 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:49.499368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 11/20] 10.222.0.50 () {56 vars in 1322 bytes} [Thu Jun 30 15:36:49 2022] GET /identity/v3/role_assignments?user.id=3a26439857af474a895fe8556a7e3453&scope.domain.id=default&role.id=5c802c0e130549e2ba423dbef846a513 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:36:49.508305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 10/21] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:36:51.655594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b51b1-446f-404a-bbeb-5fbbf99fe25a None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:51.656633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b51b1-446f-404a-bbeb-5fbbf99fe25a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:51.657112 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b51b1-446f-404a-bbeb-5fbbf99fe25a None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:51.658694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 12/22] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:51.664275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a22f-2765-45dd-9997-ade2fef3611c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:51.665105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a22f-2765-45dd-9997-ade2fef3611c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:51.665570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645a22f-2765-45dd-9997-ade2fef3611c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:51.683746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6645a22f-2765-45dd-9997-ade2fef3611c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:51.685590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6645a22f-2765-45dd-9997-ade2fef3611c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:51.767454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 11/23] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:36:51.812420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b51b1-446f-404a-bbeb-5fbbf99fe25a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:51.854734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c551404-4fd3-418e-91eb-b608066da1bf None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:51.855043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c551404-4fd3-418e-91eb-b608066da1bf None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:51.855287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c551404-4fd3-418e-91eb-b608066da1bf None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:51.857146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 12/24] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:36:51.862454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054882-f68f-437a-bec4-d6176988c22e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:51.862745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054882-f68f-437a-bec4-d6176988c22e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:51.863023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054882-f68f-437a-bec4-d6176988c22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:51.888650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5c054882-f68f-437a-bec4-d6176988c22e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:51.889871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5c054882-f68f-437a-bec4-d6176988c22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:51.937837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b51b1-446f-404a-bbeb-5fbbf99fe25a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:51.958656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:51.983325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 13/25] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:36:52.016020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c054882-f68f-437a-bec4-d6176988c22e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:52.028509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.029541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.029971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.030397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.072960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:36:52.093291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:36:52.094483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:36:52.095270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:36:52.098025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 13/26] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:36:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:36:52.108172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:52.124194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:52.187244 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c054882-f68f-437a-bec4-d6176988c22e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:52.251777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feaba1a-2719-41b0-aae7-bc60cfa4cb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.263047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaba1a-2719-41b0-aae7-bc60cfa4cb03 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.263047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaba1a-2719-41b0-aae7-bc60cfa4cb03 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.263047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feaba1a-2719-41b0-aae7-bc60cfa4cb03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.280013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdfe47-f591-4772-a9de-6b5a3b190738 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:52.290151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feaba1a-2719-41b0-aae7-bc60cfa4cb03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:52.291666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feaba1a-2719-41b0-aae7-bc60cfa4cb03 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-1feaba1a-2719-41b0-aae7-bc60cfa4cb03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:36:52.310027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:36:52.310896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:36:52.311723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:36:52.311723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:36:52.311723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:36:52.311723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:36:52.311723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 14/27] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:36:52.312527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e520aaa-3d96-4859-8723-7e0236088d66 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.313056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e520aaa-3d96-4859-8723-7e0236088d66 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.313530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e520aaa-3d96-4859-8723-7e0236088d66 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.319486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 15/28] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:52.321255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7fa38-8913-4574-97da-a852539939f1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.321867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7fa38-8913-4574-97da-a852539939f1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.322435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7fa38-8913-4574-97da-a852539939f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.341838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-bab7fa38-8913-4574-97da-a852539939f1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:52.343402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bab7fa38-8913-4574-97da-a852539939f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:52.353597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.354294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.354706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.354994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.355771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:36:52.356113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:52.357632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:52.376414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 14/29] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:52.376605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:52.377220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:52.421026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 16/30] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:36:52.422209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7fa38-8913-4574-97da-a852539939f1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:52.490425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:52.534723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5a0b6-4a22-449d-9c5e-b28c527cf5b7 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:52.539526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7fa38-8913-4574-97da-a852539939f1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:52.576870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc3d0e-06f3-4594-ba5d-e55451b59e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.577968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3d0e-06f3-4594-ba5d-e55451b59e06 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.578451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3d0e-06f3-4594-ba5d-e55451b59e06 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.578942 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc3d0e-06f3-4594-ba5d-e55451b59e06 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.579875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc3d0e-06f3-4594-ba5d-e55451b59e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:36:52.580479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc3d0e-06f3-4594-ba5d-e55451b59e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:52.582099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc3d0e-06f3-4594-ba5d-e55451b59e06 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:52.585798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.586676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.587239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.587703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] PATH_INFO: `/v3/users/3a26439857af474a895fe8556a7e3453` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.589544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] RBAC: Authorizing `identity:get_user(user_id=3a26439857af474a895fe8556a7e3453)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:52.591240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:52.593231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 17/31] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:36:52 2022] GET /identity/v3/users/3a26439857af474a895fe8556a7e3453 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:52.594165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:52.606510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 15/32] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:52.608089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550adfa-3494-4ece-a4bf-8e9558849ca3 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.608587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550adfa-3494-4ece-a4bf-8e9558849ca3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.609082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550adfa-3494-4ece-a4bf-8e9558849ca3 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.614597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 16/33] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:52.615614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550adfa-3494-4ece-a4bf-8e9558849ca3 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:52.628705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:52.704956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875ee7a-c352-4b7c-b5ed-69634f81ef37 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:52.722110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550adfa-3494-4ece-a4bf-8e9558849ca3 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:52.741984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bfb57-5382-4741-83ca-1eeba7268ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.742987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bfb57-5382-4741-83ca-1eeba7268ba3 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.743450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bfb57-5382-4741-83ca-1eeba7268ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.743858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bfb57-5382-4741-83ca-1eeba7268ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.744851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bfb57-5382-4741-83ca-1eeba7268ba3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:52.746635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07bfb57-5382-4741-83ca-1eeba7268ba3 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:52.750519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:52.761734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.762771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.763164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.763528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.764927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:52.766630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:52.773597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 17/34] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:52.774510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:52.883509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32eb6cb-08aa-4497-82a3-726cc677e5b5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:52.892282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 18/35] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Jun 30 15:36:52 2022] POST /identity/v3/roles => generated 219 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:36:52.902513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7487fe-e6ce-48ea-926a-8cd33b9eb7b8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.902916 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7487fe-e6ce-48ea-926a-8cd33b9eb7b8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.903382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7487fe-e6ce-48ea-926a-8cd33b9eb7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.929886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:52.930579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:52.930853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:52.931126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:52.931863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:52.936798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ed7487fe-e6ce-48ea-926a-8cd33b9eb7b8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:52.938767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:52.941478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ed7487fe-e6ce-48ea-926a-8cd33b9eb7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:53.106722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 18/36] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:36:52 2022] POST /identity/v3/projects => generated 301 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:36:53.107626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:53.114749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 19/37] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:36:53.164905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7487fe-e6ce-48ea-926a-8cd33b9eb7b8 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:53.210922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:53.305778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:53.377149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a08980-e8fa-4c98-bff3-9b638a244078 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:53.395422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7487fe-e6ce-48ea-926a-8cd33b9eb7b8 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:53.427343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea17a1d-6b33-4bc6-99a0-ba7f20cb3879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:53.428786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea17a1d-6b33-4bc6-99a0-ba7f20cb3879 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.429617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea17a1d-6b33-4bc6-99a0-ba7f20cb3879 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.430359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea17a1d-6b33-4bc6-99a0-ba7f20cb3879 None admin] PATH_INFO: `/v3/domains/default/users/3a26439857af474a895fe8556a7e3453/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.433855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea17a1d-6b33-4bc6-99a0-ba7f20cb3879 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3a26439857af474a895fe8556a7e3453, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:53.435464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea17a1d-6b33-4bc6-99a0-ba7f20cb3879 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:53.442139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd02eb8-6924-4d17-a2da-2dbdbd744d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:53.443205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd02eb8-6924-4d17-a2da-2dbdbd744d5c None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.443496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd02eb8-6924-4d17-a2da-2dbdbd744d5c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.443849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd02eb8-6924-4d17-a2da-2dbdbd744d5c None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.444689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd02eb8-6924-4d17-a2da-2dbdbd744d5c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:53.446430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd02eb8-6924-4d17-a2da-2dbdbd744d5c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:53.462697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 19/38] 10.222.0.50 () {58 vars in 1361 bytes} [Thu Jun 30 15:36:53 2022] PUT /identity/v3/domains/default/users/3a26439857af474a895fe8556a7e3453/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:36:53.464596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8d0d7-253e-4e6a-8c74-9757388e95b4 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.465517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8d0d7-253e-4e6a-8c74-9757388e95b4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.465978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8d0d7-253e-4e6a-8c74-9757388e95b4 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.467570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 20/40] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:53.472109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 20/40] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Jun 30 15:36:53 2022] POST /identity/v3/roles => generated 225 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:36:53.482540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740029ea-a42d-4026-b64f-9d881c12e1bb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.483273 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740029ea-a42d-4026-b64f-9d881c12e1bb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.483782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740029ea-a42d-4026-b64f-9d881c12e1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.515112 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-740029ea-a42d-4026-b64f-9d881c12e1bb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:53.517704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-740029ea-a42d-4026-b64f-9d881c12e1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:53.593525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63ed44-1412-47c5-93ca-2c7b06108679 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.593843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63ed44-1412-47c5-93ca-2c7b06108679 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.594100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63ed44-1412-47c5-93ca-2c7b06108679 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.595768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 21/41] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:36:53.604593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d2fb-25ab-4446-a42a-651f845ab797 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.604941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d2fb-25ab-4446-a42a-651f845ab797 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.605160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4d2fb-25ab-4446-a42a-651f845ab797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.605489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 21/42] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:36:53.627820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-31e4d2fb-25ab-4446-a42a-651f845ab797 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:53.628836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740029ea-a42d-4026-b64f-9d881c12e1bb None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:53.629577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-31e4d2fb-25ab-4446-a42a-651f845ab797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:53.674634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:53.722158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 22/43] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:36:53.748113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4d2fb-25ab-4446-a42a-651f845ab797 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:53.756313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:53.757853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740029ea-a42d-4026-b64f-9d881c12e1bb None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:53.796371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:53.797309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.797598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.797877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.799762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:53.801793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:53.804081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 22/44] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:36:53.809353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:53.854913 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4d2fb-25ab-4446-a42a-651f845ab797 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:53.890195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da77d0e8-a0f7-4023-8b22-4e6b1417529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:53.891081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77d0e8-a0f7-4023-8b22-4e6b1417529a None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.891390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77d0e8-a0f7-4023-8b22-4e6b1417529a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.891667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da77d0e8-a0f7-4023-8b22-4e6b1417529a None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.892484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77d0e8-a0f7-4023-8b22-4e6b1417529a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:53.893891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da77d0e8-a0f7-4023-8b22-4e6b1417529a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:53.909411 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 23/45] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Jun 30 15:36:53 2022] POST /identity/v3/roles => generated 223 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:36:53.918655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea72ed5-b58b-4421-b588-41075348ec78 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:53.922549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:53.960569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2245cc-def1-41c0-bb17-eec2e3bdeb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:53.961264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2245cc-def1-41c0-bb17-eec2e3bdeb87 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:53.961539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2245cc-def1-41c0-bb17-eec2e3bdeb87 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:53.961814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2245cc-def1-41c0-bb17-eec2e3bdeb87 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:53.962551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2245cc-def1-41c0-bb17-eec2e3bdeb87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:53.964148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2245cc-def1-41c0-bb17-eec2e3bdeb87 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:53.996019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 23/46] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:36:53 2022] POST /identity/v3/projects => generated 312 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:36:55.550723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005490c-71b8-41bb-9afb-a92b68464272 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:55.551242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005490c-71b8-41bb-9afb-a92b68464272 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:55.551242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005490c-71b8-41bb-9afb-a92b68464272 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:55.552597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 24/47] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:55.557937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf753e-12a8-4070-8038-27fc7322bb3d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:55.558334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf753e-12a8-4070-8038-27fc7322bb3d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:55.558531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf753e-12a8-4070-8038-27fc7322bb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:55.576498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6bbf753e-12a8-4070-8038-27fc7322bb3d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:55.577719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6bbf753e-12a8-4070-8038-27fc7322bb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:55.645620 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 24/48] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:36:55.668820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005490c-71b8-41bb-9afb-a92b68464272 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:55.738106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:55.777669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005490c-71b8-41bb-9afb-a92b68464272 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:55.813608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:55.814337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:55.814622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:55.814905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:55.832053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:36:55.840907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:36:55.841821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:36:55.842652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 25/49] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:36:55.848280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf753e-12a8-4070-8038-27fc7322bb3d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:55.936043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:55.952789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf753e-12a8-4070-8038-27fc7322bb3d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:55.988940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd9dd0-32c1-4749-ba62-242528f611c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:55.989617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd9dd0-32c1-4749-ba62-242528f611c0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:55.989899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd9dd0-32c1-4749-ba62-242528f611c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:55.990173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd9dd0-32c1-4749-ba62-242528f611c0 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:55.990930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd9dd0-32c1-4749-ba62-242528f611c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:36:55.991234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd9dd0-32c1-4749-ba62-242528f611c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:55.992673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd9dd0-32c1-4749-ba62-242528f611c0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:56.001203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 25/50] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:56.007354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:56.016615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0e11-aee5-42d1-a219-e64064b26033 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:56.016766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0e11-aee5-42d1-a219-e64064b26033 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:56.016991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0e11-aee5-42d1-a219-e64064b26033 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:56.018353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 26/51] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:36:56.023464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536529b-e264-44cc-a766-4e659d8e5395 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:56.023757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536529b-e264-44cc-a766-4e659d8e5395 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:56.024059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536529b-e264-44cc-a766-4e659d8e5395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:56.043783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7536529b-e264-44cc-a766-4e659d8e5395 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:56.045046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7536529b-e264-44cc-a766-4e659d8e5395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:56.112642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e44805-083c-44bc-8985-4cded4cecf99 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:56.113443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 27/52] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:36:56.137146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536529b-e264-44cc-a766-4e659d8e5395 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:56.138027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:56.147877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:56.148793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:56.149079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:56.149343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] PATH_INFO: `/v3/users/3a26439857af474a895fe8556a7e3453` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:56.150549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] RBAC: Authorizing `identity:get_user(user_id=3a26439857af474a895fe8556a7e3453)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:56.152078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:56.153728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 26/53] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:36:56 2022] GET /identity/v3/users/3a26439857af474a895fe8556a7e3453 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:56.159712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:56.219533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:56.242010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536529b-e264-44cc-a766-4e659d8e5395 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:56.262793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ec260-5106-471d-bf52-3c0d272b49fc None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:56.276462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:56.277263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:56.277544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:56.277817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:56.278985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:56.280505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:56.282438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 28/54] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:36:56.288131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:56.300770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:56.301591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:56.301858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:56.302125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:56.303341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:56.304813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:56.306457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 27/55] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:56.312364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:56.374591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:56.391870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74130462-499f-4477-9ce4-4f29db56a60e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:56.429546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c64453-0023-49dd-8cae-3effd5fdddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:56.430233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64453-0023-49dd-8cae-3effd5fdddca None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:56.430520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64453-0023-49dd-8cae-3effd5fdddca None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:56.430794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c64453-0023-49dd-8cae-3effd5fdddca None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:56.431534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c64453-0023-49dd-8cae-3effd5fdddca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:56.433170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c64453-0023-49dd-8cae-3effd5fdddca None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:56.456143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:56.476038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 29/56] 10.222.0.50 () {60 vars in 1140 bytes} [Thu Jun 30 15:36:56 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:36:56.528550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe42909-facd-4d08-9b76-540e19191f3c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:56.566577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d981b4-c401-4b95-aedd-8c7bbea3b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:56.567250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d981b4-c401-4b95-aedd-8c7bbea3b751 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:56.567525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d981b4-c401-4b95-aedd-8c7bbea3b751 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:56.567798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d981b4-c401-4b95-aedd-8c7bbea3b751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:56.568687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d981b4-c401-4b95-aedd-8c7bbea3b751 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.domain.id=default, user.id=3a26439857af474a895fe8556a7e3453) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:36:56.569000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d981b4-c401-4b95-aedd-8c7bbea3b751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:56.570505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d981b4-c401-4b95-aedd-8c7bbea3b751 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:56.580930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 28/57] 10.222.0.50 () {56 vars in 1322 bytes} [Thu Jun 30 15:36:56 2022] GET /identity/v3/role_assignments?user.id=3a26439857af474a895fe8556a7e3453&scope.domain.id=default&role.id=5c802c0e130549e2ba423dbef846a513 => generated 530 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:36:58.504554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65f3f-f532-4e8f-9fcf-11be46cae424 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:58.505384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65f3f-f532-4e8f-9fcf-11be46cae424 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:58.505517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65f3f-f532-4e8f-9fcf-11be46cae424 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:58.507718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 30/58] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:36:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:36:58.513558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc69c0-76f5-4d6e-9d81-353cf467faa2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:58.513989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc69c0-76f5-4d6e-9d81-353cf467faa2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:58.514180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc69c0-76f5-4d6e-9d81-353cf467faa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:58.533721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-35cc69c0-76f5-4d6e-9d81-353cf467faa2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:36:58.534867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-35cc69c0-76f5-4d6e-9d81-353cf467faa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:36:58.606162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 29/59] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:36:58.631141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d65f3f-f532-4e8f-9fcf-11be46cae424 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:58.737867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d65f3f-f532-4e8f-9fcf-11be46cae424 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:58.772728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12df25-3110-407b-a1fa-c214a17c70ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:58.773535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12df25-3110-407b-a1fa-c214a17c70ee None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:58.773831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12df25-3110-407b-a1fa-c214a17c70ee None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:58.774102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12df25-3110-407b-a1fa-c214a17c70ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:58.775368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12df25-3110-407b-a1fa-c214a17c70ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:58.776932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12df25-3110-407b-a1fa-c214a17c70ee None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:58.778553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 31/60] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:36:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:36:58.787085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc69c0-76f5-4d6e-9d81-353cf467faa2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:36:58.852218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:36:58.900530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc69c0-76f5-4d6e-9d81-353cf467faa2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:36:58.944996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e8bfe-3e06-4d55-89b4-7940fb3fbc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:36:58.945662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e8bfe-3e06-4d55-89b4-7940fb3fbc9d None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:36:58.945947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e8bfe-3e06-4d55-89b4-7940fb3fbc9d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:36:58.946228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e8bfe-3e06-4d55-89b4-7940fb3fbc9d None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:36:58.946951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e8bfe-3e06-4d55-89b4-7940fb3fbc9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:36:58.948509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e8bfe-3e06-4d55-89b4-7940fb3fbc9d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:36:58.980812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 30/61] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:36:58 2022] POST /identity/v3/users => generated 275 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:02.095173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36de51-74ae-4850-aabd-54e38fdd5949 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.096351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36de51-74ae-4850-aabd-54e38fdd5949 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.097172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36de51-74ae-4850-aabd-54e38fdd5949 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.099336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 32/62] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:02.105037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563a97-9c7d-489d-9e3a-39a5f9fee300 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.105897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563a97-9c7d-489d-9e3a-39a5f9fee300 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.106378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9563a97-9c7d-489d-9e3a-39a5f9fee300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.130979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf85d1-83c4-4750-99c8-02e2fa0f2a95 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.131600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf85d1-83c4-4750-99c8-02e2fa0f2a95 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.132122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf85d1-83c4-4750-99c8-02e2fa0f2a95 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.133707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 33/63] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:02.139148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56432189-6221-430b-9862-503101f90c70 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.139667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56432189-6221-430b-9862-503101f90c70 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.140205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56432189-6221-430b-9862-503101f90c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.148015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f9563a97-9c7d-489d-9e3a-39a5f9fee300 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:02.154507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f9563a97-9c7d-489d-9e3a-39a5f9fee300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:02.168118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-56432189-6221-430b-9862-503101f90c70 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:02.169689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-56432189-6221-430b-9862-503101f90c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:02.280906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 31/64] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:02.293602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:02.306324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 34/65] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:02.321160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9563a97-9c7d-489d-9e3a-39a5f9fee300 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:02.335405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56432189-6221-430b-9862-503101f90c70 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:02.400251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:02.465230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9563a97-9c7d-489d-9e3a-39a5f9fee300 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:02.500335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:02.501404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.501870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.502293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.513827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:02.523558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:02.524674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:02.525739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:02.530758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 32/66] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:02.537078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:02.552155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56432189-6221-430b-9862-503101f90c70 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:02.593777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1698a86-6054-4457-a51d-188413f181f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:02.595750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1698a86-6054-4457-a51d-188413f181f1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.596210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1698a86-6054-4457-a51d-188413f181f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.596626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1698a86-6054-4457-a51d-188413f181f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.610771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1698a86-6054-4457-a51d-188413f181f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-e1698a86-6054-4457-a51d-188413f181f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:02.622169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:02.623276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:02.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:02.626595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 35/67] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:02.628424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4966be9-4147-4e29-bcd8-ad3c2632d4d2 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.628874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4966be9-4147-4e29-bcd8-ad3c2632d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.629911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4966be9-4147-4e29-bcd8-ad3c2632d4d2 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.631742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 36/68] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:02.632683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4966be9-4147-4e29-bcd8-ad3c2632d4d2 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:02.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb4762-d8e5-4b80-a065-a5260b67c313 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:02.679856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:02.695824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:02.721177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b059fd-1c14-4b03-afd8-d687f8c913aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:02.721867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b059fd-1c14-4b03-afd8-d687f8c913aa None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.722223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b059fd-1c14-4b03-afd8-d687f8c913aa None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.722483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b059fd-1c14-4b03-afd8-d687f8c913aa None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.723176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b059fd-1c14-4b03-afd8-d687f8c913aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:02.723487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b059fd-1c14-4b03-afd8-d687f8c913aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:02.724937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b059fd-1c14-4b03-afd8-d687f8c913aa None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:02.742301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 33/69] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:02.742301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9bd2a-defc-4f8c-860b-918da67f2d85 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.742301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9bd2a-defc-4f8c-860b-918da67f2d85 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.742301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9bd2a-defc-4f8c-860b-918da67f2d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.744482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4966be9-4147-4e29-bcd8-ad3c2632d4d2 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:02.759848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-14c9bd2a-defc-4f8c-860b-918da67f2d85 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:02.761708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-14c9bd2a-defc-4f8c-860b-918da67f2d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:02.781428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa6ac5c-f39c-4e3e-8b2a-37c950518054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:02.782109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6ac5c-f39c-4e3e-8b2a-37c950518054 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.782389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6ac5c-f39c-4e3e-8b2a-37c950518054 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.782665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa6ac5c-f39c-4e3e-8b2a-37c950518054 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.783410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6ac5c-f39c-4e3e-8b2a-37c950518054 None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:02.783715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6ac5c-f39c-4e3e-8b2a-37c950518054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:02.785133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa6ac5c-f39c-4e3e-8b2a-37c950518054 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:02.793366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 37/70] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:02.798192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74525-7ae1-4f3e-8f7c-9b87c4665b91 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.798192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74525-7ae1-4f3e-8f7c-9b87c4665b91 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.798192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74525-7ae1-4f3e-8f7c-9b87c4665b91 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.798192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 38/71] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:02.798192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d74525-7ae1-4f3e-8f7c-9b87c4665b91 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:02.841344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 34/72] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:02.842112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9bd2a-defc-4f8c-860b-918da67f2d85 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:02.924274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d74525-7ae1-4f3e-8f7c-9b87c4665b91 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:02.940917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:02.946976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:02.954623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9bd2a-defc-4f8c-860b-918da67f2d85 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:02.957989 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b85d5-346a-47fd-a4f8-1373e1179a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:02.958873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b85d5-346a-47fd-a4f8-1373e1179a63 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.959157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b85d5-346a-47fd-a4f8-1373e1179a63 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.959433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b85d5-346a-47fd-a4f8-1373e1179a63 None admin] PATH_INFO: `/v3/users/3a26439857af474a895fe8556a7e3453` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.960652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b85d5-346a-47fd-a4f8-1373e1179a63 None admin] RBAC: Authorizing `identity:get_user(user_id=3a26439857af474a895fe8556a7e3453)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:02.962134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b85d5-346a-47fd-a4f8-1373e1179a63 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:02.963790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 39/73] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:02 2022] GET /identity/v3/users/3a26439857af474a895fe8556a7e3453 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:02.968933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec73e0-4073-4335-8f94-080c7ff7c9b9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.969208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec73e0-4073-4335-8f94-080c7ff7c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.969487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec73e0-4073-4335-8f94-080c7ff7c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:02.985335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d2ec73e0-4073-4335-8f94-080c7ff7c9b9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:02.986545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d2ec73e0-4073-4335-8f94-080c7ff7c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:02.989682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:02.990777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:02.991059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:02.991334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.006462 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.007937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.009708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 35/74] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:02 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:03.010476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.054688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 40/75] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:03.056387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabfbdf-f355-489b-a1c5-3e3690d340bf None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.056695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabfbdf-f355-489b-a1c5-3e3690d340bf None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.056950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabfbdf-f355-489b-a1c5-3e3690d340bf None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.058394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 41/76] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:03.058914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabfbdf-f355-489b-a1c5-3e3690d340bf None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.115550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d5e34-1ffe-4a48-ba80-7c2b5ad98d9f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.152640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.153530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.153866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.154175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.164840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:03.168765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabfbdf-f355-489b-a1c5-3e3690d340bf None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:03.173029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:03.173890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:03.174684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 36/77] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:03.175326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.185628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.195057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.205318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77026ad-2c54-4c5d-82dc-28e98add44a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.205979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77026ad-2c54-4c5d-82dc-28e98add44a8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.206270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77026ad-2c54-4c5d-82dc-28e98add44a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.206546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77026ad-2c54-4c5d-82dc-28e98add44a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.276438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77026ad-2c54-4c5d-82dc-28e98add44a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.277954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77026ad-2c54-4c5d-82dc-28e98add44a8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.280450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615a265-8a77-46bf-a2e2-fd1e45fefc9d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.314932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.315826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.316150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.316431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.374814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 42/78] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:03.376220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394a16-82ef-45d5-bb69-e80f05783b4d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.376510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394a16-82ef-45d5-bb69-e80f05783b4d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.376791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0394a16-82ef-45d5-bb69-e80f05783b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.387372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.389041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.393179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d0394a16-82ef-45d5-bb69-e80f05783b4d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:03.394379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d0394a16-82ef-45d5-bb69-e80f05783b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:03.404012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.405173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.464127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 43/79] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:03.464984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0394a16-82ef-45d5-bb69-e80f05783b4d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.494087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 37/80] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:03.495263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.562367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0394a16-82ef-45d5-bb69-e80f05783b4d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.595421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13b5f7-f098-4b28-bede-21fe6c243d3e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.601535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.602343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.602626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.602900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.604128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.605612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.607362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 44/81] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:03.608191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.610654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.626689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.633218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.633693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.633977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.634252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.634994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:03.635298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.637077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.645135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 38/82] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:03.645135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.708144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2b508-feb0-4792-ab33-76e23eafc2d9 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.743911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.744735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.745015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.745286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.746468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.748051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.749806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 45/83] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:03.750757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.753145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9117745-cfd7-48cf-b983-14dd8d21258c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.788333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.789145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.789450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.789730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.800177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:03.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:03.808750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:03.809503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 39/84] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:03.810248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.835033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.844788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:03.854257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bacfb-1804-4b69-a23b-85b92df17d1b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.889095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.889957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.890239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.890509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.891706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.893221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.894937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 46/85] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:03.895693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.914950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3220e35-6f34-4fe4-89fd-3b8330ac944b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:03.948013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:03.948717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:03.948998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:03.949292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:03.949992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:03.951525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:03.973470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 40/86] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:03 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:03.974280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:03.999679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88f873-ddc3-4c0b-b143-c55588f03517 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.038910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.039842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.040161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.040439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.041719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.043480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.049810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 47/87] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:04.049810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:04.069006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:04.089009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:04.102287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869cf1-0ebb-4fe3-80b6-de658d89aad1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.142833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.144252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.144537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.144813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.145674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=3a26439857af474a895fe8556a7e3453) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:04.146372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.147501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.154635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 41/88] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:03 2022] GET /identity/v3/role_assignments?user.id=3a26439857af474a895fe8556a7e3453&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:04.155509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:04.167465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07818710-5795-4ca2-a6fb-07925fba1276 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.220778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.221465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.221753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.222031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.222798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:04.223112 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.234633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 48/89] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:04.235356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:04.297946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51343a-262d-4e2a-8f4f-683df5577d79 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.345402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.346085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.346372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.346655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.348110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=563a6491794745e8b41b36d0b205e9df) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:04.348435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.350110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.357885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 42/90] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:04 2022] GET /identity/v3/role_assignments?user.id=563a6491794745e8b41b36d0b205e9df&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:04.358779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:04.372128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:04.389406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:04.395254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6577144-f376-4278-9189-ae8dba5cd54e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.439289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.440457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.440926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.441806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.504862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15c808-1faf-457b-bd92-8455e0fa86e3 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.527910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.529636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.552847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.554830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.555398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.555851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.558355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.560972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.563947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 43/91] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:04 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:04.574526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:04.628145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:04.656017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 49/92] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:04.657429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:04.701686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:04.722257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c082e-0747-4fd1-ba6b-c5a33f6cff0d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.797048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.797640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.797922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.798192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.817018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51864fce-3ef8-47cf-9cae-ace190c238e6 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:04.868885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:04.869773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:04.870053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:04.870325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:04.871609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.874051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.888870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 50/93] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:04 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:04.894728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:04.961194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:04.962705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:04.985806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:05.059350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:05.078531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 44/94] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:05.084113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:05.098159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21c10a-05a3-4d22-8eaf-bc34610ef4c0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:05.180957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:05.181535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:05.181819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:05.182094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:05.182940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] RBAC: Adding query filter params (role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=563a6491794745e8b41b36d0b205e9df) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:05.183252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:05.185014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:05.201634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 51/95] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:04 2022] GET /identity/v3/role_assignments?user.id=563a6491794745e8b41b36d0b205e9df&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:05.221917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f321f0b-6fb0-4837-8b5d-dc39b86289f6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:05.259820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709a0f54-d34c-4eab-b974-8c312702bebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:05.260734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a0f54-d34c-4eab-b974-8c312702bebe None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:05.261364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a0f54-d34c-4eab-b974-8c312702bebe None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:05.261638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a0f54-d34c-4eab-b974-8c312702bebe None admin] PATH_INFO: `/v3/projects/dc108e39edfd4444b015c5d68c9d3fff` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:05.262981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a0f54-d34c-4eab-b974-8c312702bebe None admin] RBAC: Authorizing `identity:get_project(project_id=dc108e39edfd4444b015c5d68c9d3fff)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:05.266218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709a0f54-d34c-4eab-b974-8c312702bebe None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:05.266442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 45/96] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:05 2022] GET /identity/v3/projects/dc108e39edfd4444b015c5d68c9d3fff => generated 312 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:05.272538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:05.383569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc9ef1-18b0-46a3-bd8a-b018e5f06b8d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:05.398393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:05.421162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60eee90-8fd1-453b-b365-9d9506402a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:05.421842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eee90-8fd1-453b-b365-9d9506402a34 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:05.422126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eee90-8fd1-453b-b365-9d9506402a34 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:05.422402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60eee90-8fd1-453b-b365-9d9506402a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:05.423251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60eee90-8fd1-453b-b365-9d9506402a34 None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=dc108e39edfd4444b015c5d68c9d3fff, user.id=563a6491794745e8b41b36d0b205e9df) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:05.423565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60eee90-8fd1-453b-b365-9d9506402a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:05.425260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60eee90-8fd1-453b-b365-9d9506402a34 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:05.432045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 52/97] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:05 2022] GET /identity/v3/role_assignments?user.id=563a6491794745e8b41b36d0b205e9df&scope.project.id=dc108e39edfd4444b015c5d68c9d3fff&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:06.819711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aac6f-74d2-4817-bbe3-e275e4ae022a None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:06.820268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aac6f-74d2-4817-bbe3-e275e4ae022a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:06.820268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aac6f-74d2-4817-bbe3-e275e4ae022a None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:06.821686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 46/98] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:06.829555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed729ab-cd9e-4d1c-a7f6-eb0fc462579f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:06.830440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed729ab-cd9e-4d1c-a7f6-eb0fc462579f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:06.830919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed729ab-cd9e-4d1c-a7f6-eb0fc462579f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:06.853433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-aed729ab-cd9e-4d1c-a7f6-eb0fc462579f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:06.854979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-aed729ab-cd9e-4d1c-a7f6-eb0fc462579f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:06.930858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 53/99] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:06.965353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aac6f-74d2-4817-bbe3-e275e4ae022a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:07.029583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:07.085068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aac6f-74d2-4817-bbe3-e275e4ae022a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:07.125971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb2d5d-5390-43a3-a84e-f1c5b345c85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:07.127165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb2d5d-5390-43a3-a84e-f1c5b345c85f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.127662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb2d5d-5390-43a3-a84e-f1c5b345c85f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.128185 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb2d5d-5390-43a3-a84e-f1c5b345c85f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.142749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb2d5d-5390-43a3-a84e-f1c5b345c85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-dccb2d5d-5390-43a3-a84e-f1c5b345c85f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:07.150753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:07.151625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:07.152422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:07.155222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 47/100] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:07.161222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed729ab-cd9e-4d1c-a7f6-eb0fc462579f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:07.300637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed729ab-cd9e-4d1c-a7f6-eb0fc462579f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:07.314849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:07.322344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf79fc0-e42d-4489-8120-642f6abf4c9d None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.323018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf79fc0-e42d-4489-8120-642f6abf4c9d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.323814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf79fc0-e42d-4489-8120-642f6abf4c9d None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.325473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 48/101] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:07.335173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dd28e-413d-4089-b690-c66ef627c692 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.335703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dd28e-413d-4089-b690-c66ef627c692 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.336200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dd28e-413d-4089-b690-c66ef627c692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.369237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:07.369931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.370214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.370497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.371230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b72dd28e-413d-4089-b690-c66ef627c692 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:07.371335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:07.371555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:07.373044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:07.373246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b72dd28e-413d-4089-b690-c66ef627c692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:07.393425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 54/102] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:07.399182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:07.468234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 49/103] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 30 15:37:07.473570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9856f2-6a1c-494b-9ed4-069ce554865d None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.474188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9856f2-6a1c-494b-9ed4-069ce554865d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.474663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9856f2-6a1c-494b-9ed4-069ce554865d None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.476371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 50/104] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:07.485662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1d5e3-03c5-4aec-babb-a98fa50e46cf None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.486204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1d5e3-03c5-4aec-babb-a98fa50e46cf None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.486697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1d5e3-03c5-4aec-babb-a98fa50e46cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.515744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a9e1d5e3-03c5-4aec-babb-a98fa50e46cf None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:07.517427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a9e1d5e3-03c5-4aec-babb-a98fa50e46cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:07.566084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:07.583119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603bbef-f8c8-40d2-8d19-db0d5db25e6c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:07.597907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 51/105] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 30 15:37:07.599095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1d5e3-03c5-4aec-babb-a98fa50e46cf None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:07.621528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7fbaf-b14a-4b3d-9635-82e16efc56f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:07.622757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7fbaf-b14a-4b3d-9635-82e16efc56f5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.623150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7fbaf-b14a-4b3d-9635-82e16efc56f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.623518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7fbaf-b14a-4b3d-9635-82e16efc56f5 None admin] PATH_INFO: `/v3/users/3a26439857af474a895fe8556a7e3453` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.625055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7fbaf-b14a-4b3d-9635-82e16efc56f5 None admin] RBAC: Authorizing `identity:get_user(user_id=3a26439857af474a895fe8556a7e3453)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:07.626507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7fbaf-b14a-4b3d-9635-82e16efc56f5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:07.628495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 55/106] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:07 2022] GET /identity/v3/users/3a26439857af474a895fe8556a7e3453 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:07.630085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444f9ff-dbe1-493d-8a38-bc4512bd7c65 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.630470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444f9ff-dbe1-493d-8a38-bc4512bd7c65 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.630842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444f9ff-dbe1-493d-8a38-bc4512bd7c65 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.632381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 56/107] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:07.636822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444f9ff-dbe1-493d-8a38-bc4512bd7c65 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:07.676561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:07.721865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1d5e3-03c5-4aec-babb-a98fa50e46cf None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:07.743265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444f9ff-dbe1-493d-8a38-bc4512bd7c65 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:07.760108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:07.761706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.762192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.762660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.774786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:07.777434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4154b87-7e6d-43bc-8655-de2aaa16f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:07.778305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4154b87-7e6d-43bc-8655-de2aaa16f9fc None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.778613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4154b87-7e6d-43bc-8655-de2aaa16f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.778886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4154b87-7e6d-43bc-8655-de2aaa16f9fc None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.780107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4154b87-7e6d-43bc-8655-de2aaa16f9fc None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:07.781589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4154b87-7e6d-43bc-8655-de2aaa16f9fc None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:07.783641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 57/108] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:07 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:07.784823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993796ab-5ab8-4ead-8683-5608631b3f22 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.785110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993796ab-5ab8-4ead-8683-5608631b3f22 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.785400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993796ab-5ab8-4ead-8683-5608631b3f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:07.786049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:07.786893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:07.788311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:07.790834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 52/109] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:07.792016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:07.808153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-993796ab-5ab8-4ead-8683-5608631b3f22 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:07.809354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-993796ab-5ab8-4ead-8683-5608631b3f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:07.830851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:07.884090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 58/110] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:07.884090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993796ab-5ab8-4ead-8683-5608631b3f22 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:07.910947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:07.935929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842e82a-f53c-476f-8436-2e822cb83de2 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:07.970986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:07.972377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:07.972858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:07.973331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:07.978144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993796ab-5ab8-4ead-8683-5608631b3f22 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:07.984932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:07.993695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:07.994578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:07.995366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:07.997927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 53/111] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:07.999240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.014189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.015150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.015435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.015761 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408/users/3a26439857af474a895fe8556a7e3453/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.017917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=3418ad75d020471a91b2b3cffb565408, user_id=3a26439857af474a895fe8556a7e3453, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.019593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.039231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 59/112] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:07 2022] PUT /identity/v3/projects/3418ad75d020471a91b2b3cffb565408/users/3a26439857af474a895fe8556a7e3453/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:08.040355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.114634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:08.124113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b17f9-c90c-4ab8-a254-1767f4f42cc0 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.148520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:08.155077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d12a3-63e0-4079-a187-6930cfdda2f0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.167558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.168690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.169250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.169753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.171924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:08.172463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.174227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.184535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 54/113] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:08.186026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.198337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.199213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.199791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.199791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.201307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.202937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.204734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 60/114] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:08.209004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.308560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a38b8-d41f-4725-b429-7796bd4e207d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.319391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640c7db-b144-4ecb-a257-881fd611df9b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.349144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ec49f-255a-47f9-bf49-e66a582816bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.350251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec49f-255a-47f9-bf49-e66a582816bf None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.350739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec49f-255a-47f9-bf49-e66a582816bf None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.351209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec49f-255a-47f9-bf49-e66a582816bf None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.352362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ec49f-255a-47f9-bf49-e66a582816bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:08.352872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ec49f-255a-47f9-bf49-e66a582816bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.354745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ec49f-255a-47f9-bf49-e66a582816bf None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.357463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.358404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.358742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.359022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.360359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.361947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.364144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 61/115] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:08 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:08.368843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.371575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 55/116] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:08.373635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609da38-2470-4e04-8f81-c943a4e9fed0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.373928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609da38-2470-4e04-8f81-c943a4e9fed0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.374990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609da38-2470-4e04-8f81-c943a4e9fed0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.375625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 56/117] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:08.380432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609da38-2470-4e04-8f81-c943a4e9fed0 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.403513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:08.413073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:08.483673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f98ae-cfbb-450f-822b-f9aae1a53b0b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.520177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609da38-2470-4e04-8f81-c943a4e9fed0 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.521724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.522301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.522586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.523481 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.523588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.525075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.552363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 62/118] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:37:08 2022] POST /identity/v3/projects => generated 302 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:08.553305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.565174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85194c77-b74a-4af5-8217-6c4361e531c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.566530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85194c77-b74a-4af5-8217-6c4361e531c5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.567092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85194c77-b74a-4af5-8217-6c4361e531c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.567590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85194c77-b74a-4af5-8217-6c4361e531c5 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.569397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85194c77-b74a-4af5-8217-6c4361e531c5 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.573114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85194c77-b74a-4af5-8217-6c4361e531c5 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.580049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 57/119] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:08 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:08.581562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82609512-d350-41eb-b8d0-a5cf10cd8aaa None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.581860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82609512-d350-41eb-b8d0-a5cf10cd8aaa None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.582149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82609512-d350-41eb-b8d0-a5cf10cd8aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.599963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-82609512-d350-41eb-b8d0-a5cf10cd8aaa None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:08.601333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-82609512-d350-41eb-b8d0-a5cf10cd8aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:08.672059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:08.694917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74d325-8618-4123-acc0-81e7127edbc4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.700269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 58/120] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:08.702227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35df8-7497-4b13-9c77-71c34c36558d None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.702743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35df8-7497-4b13-9c77-71c34c36558d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.703942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35df8-7497-4b13-9c77-71c34c36558d None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.706140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 59/121] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:08.707623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35df8-7497-4b13-9c77-71c34c36558d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.737512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:08.739049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503fe56-f144-4664-bb5a-78c8b180be4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.739964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fe56-f144-4664-bb5a-78c8b180be4d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.740304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fe56-f144-4664-bb5a-78c8b180be4d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.740620 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fe56-f144-4664-bb5a-78c8b180be4d None admin] PATH_INFO: `/v3/users/63e9590f8d08418eb5949bced9f2f9b4` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.756715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503fe56-f144-4664-bb5a-78c8b180be4d None admin] RBAC: Authorizing `identity:get_user(user_id=63e9590f8d08418eb5949bced9f2f9b4)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.758391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503fe56-f144-4664-bb5a-78c8b180be4d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.760483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 63/122] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:08 2022] GET /identity/v3/users/63e9590f8d08418eb5949bced9f2f9b4 => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:08.762055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34be9e1-c40d-4930-bf85-ddf45a90cbd7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.762341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34be9e1-c40d-4930-bf85-ddf45a90cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.762616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34be9e1-c40d-4930-bf85-ddf45a90cbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.782136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d34be9e1-c40d-4930-bf85-ddf45a90cbd7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:08.783357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d34be9e1-c40d-4930-bf85-ddf45a90cbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:08.859319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35df8-7497-4b13-9c77-71c34c36558d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:08.868225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 64/123] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:08.869352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34be9e1-c40d-4930-bf85-ddf45a90cbd7 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.914945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:08.916242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:08.916534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:08.916881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408/users/563a6491794745e8b41b36d0b205e9df/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:08.919195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] RBAC: Authorizing `identity:create_grant(project_id=3418ad75d020471a91b2b3cffb565408, user_id=563a6491794745e8b41b36d0b205e9df, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:08.922734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:08.943788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 60/124] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:08 2022] PUT /identity/v3/projects/3418ad75d020471a91b2b3cffb565408/users/563a6491794745e8b41b36d0b205e9df/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:08.948245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:08.984006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:09.043063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:09.046228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34be9e1-c40d-4930-bf85-ddf45a90cbd7 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:09.090697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069b2ec-d0a5-4b66-b0f9-c475d2b6ed1a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:09.128423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:09.129266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:09.129553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:09.129831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:09.150517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:09.151205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:09.151509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:09.151766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:09.155824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:09.163713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:09.164664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:09.165450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:09.166530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 65/125] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:09.167327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:09.264081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:09.265882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:09.347483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950e8a7-8fcf-4cde-9c4e-4e0e637975ec None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:09.397020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:09.408376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:09.418807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:09.419692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:09.419980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:09.420281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:09.438145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:09.446238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:09.447147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:09.447932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 66/126] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:09.448611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:09.449663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 61/127] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:09.454090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:09.621436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367c0f-565c-431d-bbb1-cc8a07b7e052 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:09.636464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec841bc-4e6a-4c77-95bb-10891a81771c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:09.692849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:09.693916 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:09.694402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:09.694877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:09.695841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:09.701391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:09.703253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:09.733203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 62/128] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:09.734365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:09.743627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:09.743627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:09.743627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:09.743627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:09.744067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:09.744067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:09.744067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:09.772160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 67/129] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:09.784013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:09.832670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:09.911739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526703-954e-41f0-87ec-ee403c75b4dd None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:09.923485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:09.951922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:09.952846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:09.953132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:09.953414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:09.956927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:09.958440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:09.960310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 63/130] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:09 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:09.966304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:10.049113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279d969-e26e-4789-824e-7d7319a65ad5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:10.115954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a427277-4bac-4d32-8e4b-8452deb53c5f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:10.125009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:10.156419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:10.160012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.160012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.160012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.160012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:10.160590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:10.162500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 68/131] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:09 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:10.163557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:10.170146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:10.173424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.173424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.173424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.173424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:10.175554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:10.176210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 64/132] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:09 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:10.178028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:10.330236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd863c0d-8208-4ae6-8733-0109f32b4245 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:10.386193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:10.387080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.387601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.388053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.389414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:10.391032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:10.393242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 65/133] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:10 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:10.393999 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:10.412599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b27efb-869e-4494-ab21-6547a2937109 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:10.439384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:10.456358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:10.491275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:10.492052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.492346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.492634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.493482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] RBAC: Adding query filter params (role.id=ed5809ab31cb49e9b7122b1605a2e052, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=63e9590f8d08418eb5949bced9f2f9b4) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:10.493888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:10.495452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:10.500251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa44ba-64b1-4684-bf97-065f49481dbe None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:10.508654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 69/134] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:10 2022] GET /identity/v3/role_assignments?user.id=63e9590f8d08418eb5949bced9f2f9b4&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=ed5809ab31cb49e9b7122b1605a2e052 => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:10.509362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:10.538251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e9fbb-9717-47b7-acf3-01ab7df97275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:10.539194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e9fbb-9717-47b7-acf3-01ab7df97275 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.539489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e9fbb-9717-47b7-acf3-01ab7df97275 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.539768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e9fbb-9717-47b7-acf3-01ab7df97275 None admin] PATH_INFO: `/v3/projects/dc108e39edfd4444b015c5d68c9d3fff` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.541063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e9fbb-9717-47b7-acf3-01ab7df97275 None admin] RBAC: Authorizing `identity:get_project(project_id=dc108e39edfd4444b015c5d68c9d3fff)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:10.542656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e9fbb-9717-47b7-acf3-01ab7df97275 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:10.544542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 66/135] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:10 2022] GET /identity/v3/projects/dc108e39edfd4444b015c5d68c9d3fff => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:10.545923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1eb63-c7ba-4244-b07d-c8a3d90f4032 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.546212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1eb63-c7ba-4244-b07d-c8a3d90f4032 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.546493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1eb63-c7ba-4244-b07d-c8a3d90f4032 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.547810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 67/136] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:10.552633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1eb63-c7ba-4244-b07d-c8a3d90f4032 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:10.665704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1eb63-c7ba-4244-b07d-c8a3d90f4032 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:10.677850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:10.708182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a50a8-8122-4619-9174-0b41ab94070d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:10.708363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f1670-6f50-41d0-867c-d63ce9b20285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:10.709182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1670-6f50-41d0-867c-d63ce9b20285 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.709751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1670-6f50-41d0-867c-d63ce9b20285 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.710041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f1670-6f50-41d0-867c-d63ce9b20285 None admin] PATH_INFO: `/v3/projects/dc108e39edfd4444b015c5d68c9d3fff/users/563a6491794745e8b41b36d0b205e9df/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.712988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f1670-6f50-41d0-867c-d63ce9b20285 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc108e39edfd4444b015c5d68c9d3fff, user_id=563a6491794745e8b41b36d0b205e9df, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:10.714467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f1670-6f50-41d0-867c-d63ce9b20285 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:10.734215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 68/137] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:10 2022] PUT /identity/v3/projects/dc108e39edfd4444b015c5d68c9d3fff/users/563a6491794745e8b41b36d0b205e9df/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:10.735887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ff245-1fcc-4ddc-9dff-8f2caabc3777 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.736226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ff245-1fcc-4ddc-9dff-8f2caabc3777 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.736505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ff245-1fcc-4ddc-9dff-8f2caabc3777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.758130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0d6ff245-1fcc-4ddc-9dff-8f2caabc3777 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:10.759422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0d6ff245-1fcc-4ddc-9dff-8f2caabc3777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:10.778510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:10.779426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:10.779701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:10.779969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408/users/563a6491794745e8b41b36d0b205e9df/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:10.788411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] RBAC: Authorizing `identity:create_grant(project_id=3418ad75d020471a91b2b3cffb565408, user_id=563a6491794745e8b41b36d0b205e9df, role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:10.789899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:10.819930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 70/138] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:10 2022] PUT /identity/v3/projects/3418ad75d020471a91b2b3cffb565408/users/563a6491794745e8b41b36d0b205e9df/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:10.841920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 69/139] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:10.866326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:10.918510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:10.990774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eccad26-81d7-4bbd-8b39-0783b82199db None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:11.035691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189dc753-6353-49e2-86d0-feb8ed4a7198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:11.036540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc753-6353-49e2-86d0-feb8ed4a7198 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.036821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc753-6353-49e2-86d0-feb8ed4a7198 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.037099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189dc753-6353-49e2-86d0-feb8ed4a7198 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.049154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189dc753-6353-49e2-86d0-feb8ed4a7198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-189dc753-6353-49e2-86d0-feb8ed4a7198 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:11.057052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:11.058948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:11.060488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 71/140] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:11.064989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ff245-1fcc-4ddc-9dff-8f2caabc3777 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:11.171863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:11.174738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ff245-1fcc-4ddc-9dff-8f2caabc3777 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:11.180840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b57e5-f161-463e-a1ef-1571b9a13453 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.181108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b57e5-f161-463e-a1ef-1571b9a13453 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.181391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b57e5-f161-463e-a1ef-1571b9a13453 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.183065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 72/141] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:11.188193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88c298-7dc0-4171-9110-b6c6800b186d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.189086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88c298-7dc0-4171-9110-b6c6800b186d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.189420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88c298-7dc0-4171-9110-b6c6800b186d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.215651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6f88c298-7dc0-4171-9110-b6c6800b186d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:11.217718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6f88c298-7dc0-4171-9110-b6c6800b186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:11.218177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:11.218876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.219162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.219442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.220238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:11.220552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:11.221984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:11.232062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 70/142] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:11.237481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:11.320627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 73/143] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:11.362745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88c298-7dc0-4171-9110-b6c6800b186d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:11.422961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ac3ce-f589-4594-be19-8e154a562a74 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:11.428941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:11.495156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:11.496084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.496369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.496646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] PATH_INFO: `/v3/users/3a26439857af474a895fe8556a7e3453` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.498893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] RBAC: Authorizing `identity:get_user(user_id=3a26439857af474a895fe8556a7e3453)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:11.500522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:11.502175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 71/144] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:11 2022] GET /identity/v3/users/3a26439857af474a895fe8556a7e3453 => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:11.511042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:11.570181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:11.571048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88c298-7dc0-4171-9110-b6c6800b186d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:11.647059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:11.647888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.648182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.648453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.654201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:11.655697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:11.657566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 74/145] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:11.665281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:11.722079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e345b-d53e-4f03-accf-dcd1d76bdb13 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:11.797255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:11.798008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.798297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.798578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.814859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e1e263-01ba-4a04-826c-c7cbb94bdf42 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:11.816447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:11.853816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e851d-108b-47ae-b724-21bb942a0f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:11.854601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e851d-108b-47ae-b724-21bb942a0f52 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.854940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e851d-108b-47ae-b724-21bb942a0f52 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.855295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e851d-108b-47ae-b724-21bb942a0f52 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.856114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e851d-108b-47ae-b724-21bb942a0f52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:11.858048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e851d-108b-47ae-b724-21bb942a0f52 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:11.910118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 75/146] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:11 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:11.911247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464c3a-ea81-4fc0-a7af-4dfc54620e98 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.911530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464c3a-ea81-4fc0-a7af-4dfc54620e98 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.911874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464c3a-ea81-4fc0-a7af-4dfc54620e98 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.913159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 76/147] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:11.920170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea2015-3a18-4207-a87e-c72c1c0165df None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:11.921115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea2015-3a18-4207-a87e-c72c1c0165df None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:11.921415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea2015-3a18-4207-a87e-c72c1c0165df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:11.941938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-32ea2015-3a18-4207-a87e-c72c1c0165df None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:11.943329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-32ea2015-3a18-4207-a87e-c72c1c0165df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:11.950270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:11.951888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:12.011365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:12.050759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 77/148] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:12.107977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 72/149] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:12.113713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea2015-3a18-4207-a87e-c72c1c0165df None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:12.117562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:12.231570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e4ebb-0c00-45ba-a106-2e5b96cbb609 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:12.283021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:12.296484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:12.297374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:12.297653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:12.297924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:12.304355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:12.315036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:12.324430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:12.325342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:12.326125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 73/150] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:12.331330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:12.349912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea2015-3a18-4207-a87e-c72c1c0165df None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:12.438975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d9d84-5643-4caf-835c-6e5d5175c1b1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:12.477085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:12.477772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:12.478046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:12.478319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:12.479072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:12.479375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:12.480792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:12.483947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:12.484843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:12.485120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:12.485389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:12.486577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:12.488057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:12.489774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 78/151] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:12 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:12.496293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:12.507545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 74/152] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:12.520864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:12.590216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:12.648212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a90cd-1e27-432b-9e30-a659947a7f42 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:12.689120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:12.690243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:12.690527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:12.690799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:12.692619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:12.694266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:12.696286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 75/153] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:12 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:12.702860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:12.705020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce089de1-366d-4544-9822-afe729e92137 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:12.791618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:12.792549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:12.792989 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:12.793413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:12.794611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=3a26439857af474a895fe8556a7e3453) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:12.795122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:12.796774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:12.809909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 79/154] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:12 2022] GET /identity/v3/role_assignments?user.id=3a26439857af474a895fe8556a7e3453&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:12.814407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336edd1-5d3e-4f60-9cdb-cfa2d21e46e5 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:12.836944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:12.852539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49a2d5-9e2f-447d-9834-0cba84f8d5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:12.853208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49a2d5-9e2f-447d-9834-0cba84f8d5a8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:12.853493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49a2d5-9e2f-447d-9834-0cba84f8d5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:12.853838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49a2d5-9e2f-447d-9834-0cba84f8d5a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:12.926042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49a2d5-9e2f-447d-9834-0cba84f8d5a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:12.927524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49a2d5-9e2f-447d-9834-0cba84f8d5a8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:13.030728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 76/155] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:13.036708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:13.073411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:13.157577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033bfaf-9df5-4d8a-98e4-724947df9c58 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:13.158546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033bfaf-9df5-4d8a-98e4-724947df9c58 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:13.159022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033bfaf-9df5-4d8a-98e4-724947df9c58 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:13.161268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 77/156] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:13.168622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcde21-cfc0-4cde-b5cb-74db7a2e005a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:13.170270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcde21-cfc0-4cde-b5cb-74db7a2e005a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:13.170757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcde21-cfc0-4cde-b5cb-74db7a2e005a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:13.194829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee895e6-592a-4fe4-a015-67e8155ef46d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:13.201565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-45bcde21-cfc0-4cde-b5cb-74db7a2e005a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:13.203007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-45bcde21-cfc0-4cde-b5cb-74db7a2e005a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:13.241723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:13.244157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:13.245221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:13.245652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:13.246230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:13.250895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:13.252668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:13.254667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 80/157] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:13 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:13.260504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:13.298441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 78/158] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:13.345944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcde21-cfc0-4cde-b5cb-74db7a2e005a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:13.368390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:13.381229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4f538-b42a-41ea-8fbd-8e11b8294631 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:13.422053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:13.422986 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:13.423414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:13.423948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:13.424967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=563a6491794745e8b41b36d0b205e9df) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:13.425422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:13.427094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:13.434756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 81/159] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:13 2022] GET /identity/v3/role_assignments?user.id=563a6491794745e8b41b36d0b205e9df&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:13.547594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcde21-cfc0-4cde-b5cb-74db7a2e005a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:13.591762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:13.592608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:13.592887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:13.593160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:13.604737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:13.612517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:13.613509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:13.614982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 79/160] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:13.620048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:13.757642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b959b1-eb59-425c-9da9-4a03d54644a0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:13.797775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:13.799011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:13.799498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:13.799962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:13.800998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:13.801508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:13.803098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:13.812734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 82/161] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:13.817953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:13.844448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:13.957306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f4d28-1f0e-414b-a451-4a118536556a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:14.005721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e330238-144a-467a-8b97-ad39f5d7ec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:14.006611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e330238-144a-467a-8b97-ad39f5d7ec22 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.006883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e330238-144a-467a-8b97-ad39f5d7ec22 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.007148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e330238-144a-467a-8b97-ad39f5d7ec22 None admin] PATH_INFO: `/v3/users/63e9590f8d08418eb5949bced9f2f9b4` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.011712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e330238-144a-467a-8b97-ad39f5d7ec22 None admin] RBAC: Authorizing `identity:get_user(user_id=63e9590f8d08418eb5949bced9f2f9b4)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:14.013260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e330238-144a-467a-8b97-ad39f5d7ec22 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:14.014916 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 80/162] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:13 2022] GET /identity/v3/users/63e9590f8d08418eb5949bced9f2f9b4 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:14.020573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:14.041745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08899dae-c52f-4ea5-b13a-0b1ca1ec674e None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.042049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08899dae-c52f-4ea5-b13a-0b1ca1ec674e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.042298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08899dae-c52f-4ea5-b13a-0b1ca1ec674e None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.043831 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 81/163] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:14.047714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:14.049171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0cfdc-b1f4-4af4-bfd1-28d69e3acde5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.049455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0cfdc-b1f4-4af4-bfd1-28d69e3acde5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.049769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0cfdc-b1f4-4af4-bfd1-28d69e3acde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.084029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ddd0cfdc-b1f4-4af4-bfd1-28d69e3acde5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:14.088907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ddd0cfdc-b1f4-4af4-bfd1-28d69e3acde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:14.221960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec9bed-72c3-4559-8a2f-194cf20ac223 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:14.228433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 82/164] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:14.230257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b7ff1-7d32-48e9-be0d-7473ed989d00 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.230684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b7ff1-7d32-48e9-be0d-7473ed989d00 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.231093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b7ff1-7d32-48e9-be0d-7473ed989d00 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.232663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 83/165] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:14.237724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c8018-d9e0-4cc9-ab5a-dd38036f920c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.238206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c8018-d9e0-4cc9-ab5a-dd38036f920c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.238636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c8018-d9e0-4cc9-ab5a-dd38036f920c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.255176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-117c8018-d9e0-4cc9-ab5a-dd38036f920c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:14.256659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-117c8018-d9e0-4cc9-ab5a-dd38036f920c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:14.258996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:14.302564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:14.303638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.303981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.304363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.309160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:14.310516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:14.312491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 83/166] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:14 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:14.313440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:14.334854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 84/167] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:14.335934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c8018-d9e0-4cc9-ab5a-dd38036f920c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:14.458177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c8018-d9e0-4cc9-ab5a-dd38036f920c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:14.491949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:14.493149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.493629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.494078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408/users/63e9590f8d08418eb5949bced9f2f9b4/roles/ed5809ab31cb49e9b7122b1605a2e052` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.496231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] RBAC: Authorizing `identity:create_grant(project_id=3418ad75d020471a91b2b3cffb565408, user_id=63e9590f8d08418eb5949bced9f2f9b4, role_id=ed5809ab31cb49e9b7122b1605a2e052)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:14.497879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:14.513397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 85/168] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:14 2022] PUT /identity/v3/projects/3418ad75d020471a91b2b3cffb565408/users/63e9590f8d08418eb5949bced9f2f9b4/roles/ed5809ab31cb49e9b7122b1605a2e052 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:14.513978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:14.519475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:14.532380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:14.590058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393bc71-4089-4311-afa5-81d30c11ff2f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:14.708751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:14.709577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.709859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.710134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.714620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b062bc1-89ce-4590-b799-d198baf9ae04 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:14.761669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:14.767502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:14.768387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.768667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.768938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:14.769428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:14.770286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:14.771066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 84/169] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:14.777012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:14.780061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:14.797390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:14.798552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:14.799347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 86/170] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:14.805361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:14.892418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:14.919638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6dd19-b904-48ee-8c9d-ba0b189486ea None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:14.937329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6bffe1-51da-4ee8-8671-4e4536f6e3ed None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:14.972460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:14.973434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.973889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.974315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.975276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:14.975728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:14.977333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:14.977754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:14.978016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:14.978296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:14.978566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:14.979308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:14.979616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:14.981016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:14.988590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 87/171] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:14.994650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:14.995770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 85/172] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:15.001324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:15.032439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:15.169040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91da1c-a3ad-4d16-8009-b9d621f2ff3c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:15.196321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c1a06-e486-44e7-bf3f-a7d9cbe584f6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:15.235699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:15.246312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:15.247242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:15.247551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:15.247826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:15.250199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:15.251687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:15.261626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 86/173] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:15 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:15.262529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:15.288103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:15.288103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:15.288353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:15.288582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] PATH_INFO: `/v3/users/563a6491794745e8b41b36d0b205e9df` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:15.292062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] RBAC: Authorizing `identity:get_user(user_id=563a6491794745e8b41b36d0b205e9df)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:15.293524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:15.295346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 88/174] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:14 2022] GET /identity/v3/users/563a6491794745e8b41b36d0b205e9df => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:15.302713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:15.402211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:15.436201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1895621-d52b-449a-9e95-106833ae7648 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:15.442720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599aa512-4550-4fed-8c1a-9c9fcd6ae49a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:15.471209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:15.471871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:15.472187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:15.472467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:15.497308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:15.497967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:15.498235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:15.498510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:15.531758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:15.545322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:15.546998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:15.608958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:15.610555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:15.649645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 89/175] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:15.656890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:15.768872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:15.846889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:15.852012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 87/176] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:15.857767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:15.873877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb97a5a-1299-4478-a7b3-8082bd387d1e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:15.924825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638cdea-c4b8-4d05-abbb-cefd5ae0aa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:15.926076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638cdea-c4b8-4d05-abbb-cefd5ae0aa1c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:15.926568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638cdea-c4b8-4d05-abbb-cefd5ae0aa1c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:15.927040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638cdea-c4b8-4d05-abbb-cefd5ae0aa1c None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:15.934181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638cdea-c4b8-4d05-abbb-cefd5ae0aa1c None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:15.936073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638cdea-c4b8-4d05-abbb-cefd5ae0aa1c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:15.938220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 90/177] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:15 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:15.939752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d4c94-df55-4b31-8dc3-a6707076417c None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:15.940267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d4c94-df55-4b31-8dc3-a6707076417c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:15.940758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d4c94-df55-4b31-8dc3-a6707076417c None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:15.942187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 91/178] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:15.951967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d4c94-df55-4b31-8dc3-a6707076417c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.036230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4a6f3-b60b-42b6-a5b7-e05bd34c4a59 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.082976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429ab95-6730-4e66-92fc-db512207b193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.084216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ab95-6730-4e66-92fc-db512207b193 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.084569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ab95-6730-4e66-92fc-db512207b193 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.084912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ab95-6730-4e66-92fc-db512207b193 None admin] PATH_INFO: `/v3/projects/dc108e39edfd4444b015c5d68c9d3fff` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.087030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ab95-6730-4e66-92fc-db512207b193 None admin] RBAC: Authorizing `identity:get_project(project_id=dc108e39edfd4444b015c5d68c9d3fff)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:16.089143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ab95-6730-4e66-92fc-db512207b193 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:16.091828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 88/179] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:15 2022] GET /identity/v3/projects/dc108e39edfd4444b015c5d68c9d3fff => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:16.093281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ee038-be44-4e50-847f-8f4adb4c853c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.093635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ee038-be44-4e50-847f-8f4adb4c853c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.093978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ee038-be44-4e50-847f-8f4adb4c853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.094310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d4c94-df55-4b31-8dc3-a6707076417c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.117301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-5d1ee038-be44-4e50-847f-8f4adb4c853c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:16.118933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5d1ee038-be44-4e50-847f-8f4adb4c853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:16.130647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258001d-c10d-48ac-86f1-a1aaf0a561ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.131358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258001d-c10d-48ac-86f1-a1aaf0a561ea None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.131640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258001d-c10d-48ac-86f1-a1aaf0a561ea None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.131913 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258001d-c10d-48ac-86f1-a1aaf0a561ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.132786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258001d-c10d-48ac-86f1-a1aaf0a561ea None admin] RBAC: Adding query filter params (role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=563a6491794745e8b41b36d0b205e9df) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:16.133092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258001d-c10d-48ac-86f1-a1aaf0a561ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:16.134535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258001d-c10d-48ac-86f1-a1aaf0a561ea None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:16.141016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 92/180] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:15 2022] GET /identity/v3/role_assignments?user.id=563a6491794745e8b41b36d0b205e9df&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:16.142247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac1b8e-eef5-4dac-ab1c-d1f6c3146c83 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.142662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac1b8e-eef5-4dac-ab1c-d1f6c3146c83 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.143130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac1b8e-eef5-4dac-ab1c-d1f6c3146c83 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.145236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 93/181] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:16.146520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac1b8e-eef5-4dac-ab1c-d1f6c3146c83 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.179809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:16.183042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:16.264187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 89/182] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:16.267846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa794-c847-4e31-92d2-30276a0807ea None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.268948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa794-c847-4e31-92d2-30276a0807ea None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.276028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa794-c847-4e31-92d2-30276a0807ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.307201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fd0aa794-c847-4e31-92d2-30276a0807ea None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:16.309989 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fd0aa794-c847-4e31-92d2-30276a0807ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:16.341128 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac1b8e-eef5-4dac-ab1c-d1f6c3146c83 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.386137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.386722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.387001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.387274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.388267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=dc108e39edfd4444b015c5d68c9d3fff, user.id=563a6491794745e8b41b36d0b205e9df) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:16.388579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:16.390041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:16.400457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 94/183] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/role_assignments?user.id=563a6491794745e8b41b36d0b205e9df&scope.project.id=dc108e39edfd4444b015c5d68c9d3fff&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:16.401132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.408638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 90/184] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:16.431976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aa794-c847-4e31-92d2-30276a0807ea None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.498335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:16.517293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b137d30-1e48-4ffe-81ad-a3a7f93f8f3f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.542173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:16.546124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aa794-c847-4e31-92d2-30276a0807ea None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.555273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a5b3-abb1-4e83-8110-201a41abdce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.556137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a5b3-abb1-4e83-8110-201a41abdce1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.556477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a5b3-abb1-4e83-8110-201a41abdce1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.556756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a5b3-abb1-4e83-8110-201a41abdce1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.569942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7a5b3-abb1-4e83-8110-201a41abdce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-61d7a5b3-abb1-4e83-8110-201a41abdce1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:16.578514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:16.579449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:16.580273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 95/185] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:16.581451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284bae1-cc3a-4a54-a8ac-3902813d3805 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.581451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284bae1-cc3a-4a54-a8ac-3902813d3805 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.581747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284bae1-cc3a-4a54-a8ac-3902813d3805 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.583597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 96/186] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:16.588556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948100-b184-4976-85f5-62cc653bf8d1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.588846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948100-b184-4976-85f5-62cc653bf8d1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.589126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b948100-b184-4976-85f5-62cc653bf8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.589409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.590228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.590486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.590820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.603289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:16.609478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9b948100-b184-4976-85f5-62cc653bf8d1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:16.610687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9b948100-b184-4976-85f5-62cc653bf8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:16.611076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:16.613160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:16.615056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:16.616625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 91/187] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:16.617415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.700842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 97/188] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:16.702043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948100-b184-4976-85f5-62cc653bf8d1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.747691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e7bf4-e2fb-477c-ab52-416023c684a4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.781840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.782499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.782775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.783045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.783784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:16.784117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:16.785496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:16.788190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:16.794057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 92/189] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:16.798995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.819805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b948100-b184-4976-85f5-62cc653bf8d1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.820853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:16.858099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.859093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.859657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.859948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.860791 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:16.861102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:16.862659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:16.878271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 98/190] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:16.878271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.897498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de3c48-3a1f-44b8-9bac-ece09846ee54 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:16.931704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:16.932527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:16.932810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:16.933091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:16.934264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:16.935813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:16.937682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 93/191] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:16.938282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:16.994487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3747120-9e9d-4789-8d3f-08cb1fb3e436 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.031387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.032304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.032584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.032862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] PATH_INFO: `/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.036432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.039420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.049310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34a1b2-deea-4b81-8c52-ae58fed03fa0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.049461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] RBAC: Authorizing `identity:get_user(user_id=712ebe2bd9a64fd4b5168ac1dfac183c)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.050789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.052904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 99/192] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:17.053721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.084553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.085667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.085951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.086223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] PATH_INFO: `/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.087435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] RBAC: Authorizing `identity:get_user(user_id=712ebe2bd9a64fd4b5168ac1dfac183c)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.088934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.090763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 94/193] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:16 2022] GET /identity/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:17.091403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.165208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe49a8-3fd1-41aa-989f-c2124b2bac3d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.196344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bd103-09d7-456e-85b3-de233fd0f468 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.203559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.204673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.204955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.205225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.206302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.208078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.229038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 100/194] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:17 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:17.229685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.256920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878cbfe-c186-4e10-a501-ef25ca8381f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.258813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878cbfe-c186-4e10-a501-ef25ca8381f0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.259630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878cbfe-c186-4e10-a501-ef25ca8381f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.260198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878cbfe-c186-4e10-a501-ef25ca8381f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.292599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.305824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.355860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358ce4b-693c-4da0-88a9-68f80244a0bf None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.365952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878cbfe-c186-4e10-a501-ef25ca8381f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.368641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878cbfe-c186-4e10-a501-ef25ca8381f0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.396839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.397655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.398596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.398596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.480639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 95/195] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:17.481862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9c7b0-7783-438a-8c6c-4feffac1b430 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.482151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9c7b0-7783-438a-8c6c-4feffac1b430 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.482249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.482426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9c7b0-7783-438a-8c6c-4feffac1b430 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.483832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.483937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 96/196] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:17.490898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356de9-995d-472f-b3ee-a2021e17999a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.491188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356de9-995d-472f-b3ee-a2021e17999a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.491465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5356de9-995d-472f-b3ee-a2021e17999a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.513656 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f5356de9-995d-472f-b3ee-a2021e17999a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:17.514807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f5356de9-995d-472f-b3ee-a2021e17999a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:17.539513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.581918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.600801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 97/197] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:17.601766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 101/198] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:17.601885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5356de9-995d-472f-b3ee-a2021e17999a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.609279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.715667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5356de9-995d-472f-b3ee-a2021e17999a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.739723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff50a2-80dd-4bb3-b66d-51f25bfd8cd4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.767073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.769011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.769758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.770267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.772540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.774334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.776183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.777120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.777334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.777518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 98/199] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:17.777626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.778824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.778930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.780330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.782345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 102/200] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:17.783110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.802655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.846484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:17.893050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2761077-d0dd-4521-bc5f-952986f6af86 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.911341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43e58d-a15e-4217-89fb-8054b5b407e1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:17.932411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.933579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.934049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.934564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.945440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:17.952455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:17.953121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:17.953401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:17.953672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:17.954509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] RBAC: Adding query filter params (role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=712ebe2bd9a64fd4b5168ac1dfac183c) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:17.954819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:17.955204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:17.956287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:17.957138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:17.957747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:17.959466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 99/201] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:17.960535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:17.967260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 103/202] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/role_assignments?user.id=712ebe2bd9a64fd4b5168ac1dfac183c&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:17.974173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:18.093589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:18.117976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ec5c6-e057-4a37-9094-3ac2193c9b9b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:18.158976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:18.160608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:18.161050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:18.161465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:18.162371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:18.162817 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:18.164403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:18.175443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 104/203] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:18.181453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:18.220451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ef981-024e-4dd6-afc8-dfb37704c4c6 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:18.244255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:18.276010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:18.276010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:18.276010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:18.276010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:18.276335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=712ebe2bd9a64fd4b5168ac1dfac183c) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:18.276335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:18.277482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:18.295466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 100/204] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:17 2022] GET /identity/v3/role_assignments?user.id=712ebe2bd9a64fd4b5168ac1dfac183c&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:18.325602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a141f0-5e40-4b36-a6e1-7ee4b47b1a88 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:18.363118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:18.363918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:18.364228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:18.364500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] PATH_INFO: `/v3/users/63e9590f8d08418eb5949bced9f2f9b4` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:18.365719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] RBAC: Authorizing `identity:get_user(user_id=63e9590f8d08418eb5949bced9f2f9b4)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:18.367174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:18.368877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 105/205] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:18 2022] GET /identity/v3/users/63e9590f8d08418eb5949bced9f2f9b4 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:18.374749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:18.482095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c828-c2a5-4de0-878e-7fd46cc5a574 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:18.518160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:18.518822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:18.519104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:18.519374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:18.569808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:18.590275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:18.591835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:18.690924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 101/206] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:18.696559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:18.713737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:18.801368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b9e5d-a0fd-4cc4-b674-10c21d95351c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:18.835724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bc741-761d-4a05-b0f1-94d395124192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:18.836632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc741-761d-4a05-b0f1-94d395124192 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:18.836911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc741-761d-4a05-b0f1-94d395124192 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:18.837180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bc741-761d-4a05-b0f1-94d395124192 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:18.838370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bc741-761d-4a05-b0f1-94d395124192 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:18.839809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bc741-761d-4a05-b0f1-94d395124192 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:18.841599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 106/207] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:18 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:18.847649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:18.932767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:18.947668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95faf9-5424-44d0-8299-fe7c2642468f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:18.982022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8228b-b051-4ca2-81ef-f5b25c794066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:18.982683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8228b-b051-4ca2-81ef-f5b25c794066 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:18.982961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8228b-b051-4ca2-81ef-f5b25c794066 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:18.983233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8228b-b051-4ca2-81ef-f5b25c794066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:18.984208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8228b-b051-4ca2-81ef-f5b25c794066 None admin] RBAC: Adding query filter params (role.id=ed5809ab31cb49e9b7122b1605a2e052, scope.project.id=3418ad75d020471a91b2b3cffb565408, user.id=63e9590f8d08418eb5949bced9f2f9b4) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:18.984517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8228b-b051-4ca2-81ef-f5b25c794066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:18.985952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8228b-b051-4ca2-81ef-f5b25c794066 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:18.992228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 102/208] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:18 2022] GET /identity/v3/role_assignments?user.id=63e9590f8d08418eb5949bced9f2f9b4&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=ed5809ab31cb49e9b7122b1605a2e052 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:19.446124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb53a6-1892-4ed0-95ff-f59cb00f1e47 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:19.446636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb53a6-1892-4ed0-95ff-f59cb00f1e47 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:19.446737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb53a6-1892-4ed0-95ff-f59cb00f1e47 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:19.448335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 107/209] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:19.454064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73ced-57e0-4cab-a9e9-3f77294ced89 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:19.454578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73ced-57e0-4cab-a9e9-3f77294ced89 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:19.454653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e73ced-57e0-4cab-a9e9-3f77294ced89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:19.472418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c5e73ced-57e0-4cab-a9e9-3f77294ced89 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:19.473635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c5e73ced-57e0-4cab-a9e9-3f77294ced89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:19.542842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 103/210] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:19.568472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb53a6-1892-4ed0-95ff-f59cb00f1e47 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:19.660652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:19.682513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb53a6-1892-4ed0-95ff-f59cb00f1e47 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:19.721703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecafe1-e296-4c59-bece-8a223c2c9d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:19.722513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecafe1-e296-4c59-bece-8a223c2c9d76 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:19.722797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecafe1-e296-4c59-bece-8a223c2c9d76 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:19.723079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecafe1-e296-4c59-bece-8a223c2c9d76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:19.735313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecafe1-e296-4c59-bece-8a223c2c9d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-39ecafe1-e296-4c59-bece-8a223c2c9d76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:19.743389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:19.744308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:19.745117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 108/211] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:19.750676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73ced-57e0-4cab-a9e9-3f77294ced89 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:19.833807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:19.859606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e73ced-57e0-4cab-a9e9-3f77294ced89 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:19.892556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1475f66-ab9d-446c-810d-c1e96ba96854 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:19.892852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1475f66-ab9d-446c-810d-c1e96ba96854 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:19.893113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1475f66-ab9d-446c-810d-c1e96ba96854 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:19.893512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:19.894181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:19.894460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:19.894614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 109/212] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:19.894786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:19.895601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:19.895903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:19.897324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:19.899371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84489a-2d74-4a7a-aa5c-b5d13f5ea525 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:19.899659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84489a-2d74-4a7a-aa5c-b5d13f5ea525 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:19.899934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f84489a-2d74-4a7a-aa5c-b5d13f5ea525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:19.907547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 104/213] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:19.912110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:19.917273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8f84489a-2d74-4a7a-aa5c-b5d13f5ea525 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:19.918470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8f84489a-2d74-4a7a-aa5c-b5d13f5ea525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:19.996292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 110/214] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:20.021387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84489a-2d74-4a7a-aa5c-b5d13f5ea525 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:20.024630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a0c5-3fa4-4f05-b92e-bcd8a7b4485f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:20.064966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:20.066911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.067212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.067483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] PATH_INFO: `/v3/users/f55dec7957ca45cbad2f0c164a98ff0f` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.071066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:20.084975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] RBAC: Authorizing `identity:get_user(user_id=f55dec7957ca45cbad2f0c164a98ff0f)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:20.086896 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:20.088712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 105/215] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:19 2022] GET /identity/v3/users/f55dec7957ca45cbad2f0c164a98ff0f => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:20.094472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:20.115360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:20.154866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f84489a-2d74-4a7a-aa5c-b5d13f5ea525 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:20.192074 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:20.192881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.193157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.193424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.194691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:20.196180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:20.197979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 111/216] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:20.203755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:20.221523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b4494-9d8d-4638-8e93-b0f8f730926a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:20.261751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:20.262398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.262696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.262961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.323754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240979c3-27ac-4ca5-aec0-0f24ce10dbb1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:20.338340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:20.350835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:20.353052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:20.369175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833c228-41d4-4486-8d24-1595883013ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:20.370432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c228-41d4-4486-8d24-1595883013ee None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.371154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c228-41d4-4486-8d24-1595883013ee None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.372072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833c228-41d4-4486-8d24-1595883013ee None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.373022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833c228-41d4-4486-8d24-1595883013ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:20.375190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833c228-41d4-4486-8d24-1595883013ee None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:20.387820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:20.408311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 112/217] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:20 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:37:20.461152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fa204-808a-4413-a330-798762b686cb None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.462234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fa204-808a-4413-a330-798762b686cb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.462731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fa204-808a-4413-a330-798762b686cb None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.464471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 113/218] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:20.470451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a253334-b828-4bfa-906c-b7be2fa02441 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.471070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a253334-b828-4bfa-906c-b7be2fa02441 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.471543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a253334-b828-4bfa-906c-b7be2fa02441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.480108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 106/219] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:20.485659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:20.491607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-2a253334-b828-4bfa-906c-b7be2fa02441 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:20.493183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2a253334-b828-4bfa-906c-b7be2fa02441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:20.590418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 114/220] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:20.595639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:20.603943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4e558-6313-4b71-a1b3-91853351fa4b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:20.614248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a253334-b828-4bfa-906c-b7be2fa02441 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:20.646146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:20.647350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.648070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.648512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.650431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:20.652380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:20.655267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 107/221] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:20 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:20.661218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:20.746483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a253334-b828-4bfa-906c-b7be2fa02441 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:20.783936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:20.786699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0dbfb-fa89-4222-87a1-2a2cef04e812 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:20.798048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aae3a7-f2c6-49ed-8d69-7d247a3fbaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:20.798912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aae3a7-f2c6-49ed-8d69-7d247a3fbaa5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.799162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aae3a7-f2c6-49ed-8d69-7d247a3fbaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.799443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aae3a7-f2c6-49ed-8d69-7d247a3fbaa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.815804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aae3a7-f2c6-49ed-8d69-7d247a3fbaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-b9aae3a7-f2c6-49ed-8d69-7d247a3fbaa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:20.823837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:20.824776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:20.825836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:20.828135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 115/222] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:20.828537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7e6b6-841a-43f9-9fc7-fa9a1e38b045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:20.829254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e6b6-841a-43f9-9fc7-fa9a1e38b045 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.829534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e6b6-841a-43f9-9fc7-fa9a1e38b045 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.829855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7e6b6-841a-43f9-9fc7-fa9a1e38b045 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.830560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cfa01-d1bd-4252-a30b-14ffd526ae36 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.830776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e6b6-841a-43f9-9fc7-fa9a1e38b045 None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=f55dec7957ca45cbad2f0c164a98ff0f) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:20.831087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e6b6-841a-43f9-9fc7-fa9a1e38b045 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:20.831444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cfa01-d1bd-4252-a30b-14ffd526ae36 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.831911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cfa01-d1bd-4252-a30b-14ffd526ae36 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.832649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7e6b6-841a-43f9-9fc7-fa9a1e38b045 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:20.833724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 116/223] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:20.834962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cfa01-d1bd-4252-a30b-14ffd526ae36 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:20.839095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 108/224] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:20 2022] GET /identity/v3/role_assignments?user.id=f55dec7957ca45cbad2f0c164a98ff0f&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:20.840783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdf0ce-9e22-4565-9548-3c891dd46339 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:20.844100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdf0ce-9e22-4565-9548-3c891dd46339 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:20.844731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdf0ce-9e22-4565-9548-3c891dd46339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:20.882447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a5cdf0ce-9e22-4565-9548-3c891dd46339 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:20.884071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a5cdf0ce-9e22-4565-9548-3c891dd46339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:20.944100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:20.975371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cfa01-d1bd-4252-a30b-14ffd526ae36 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:21.015088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:21.017026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:21.017026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:21.017026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:21.017255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:21.017519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:21.018861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:21.033156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 117/225] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:21.034841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:21.038256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 109/226] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:21.061368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdf0ce-9e22-4565-9548-3c891dd46339 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:21.102948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:21.183692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a68395-3985-4f1f-9528-6417d3633fe0 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:21.217532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cdf0ce-9e22-4565-9548-3c891dd46339 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:21.218026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:21.218917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:21.219195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:21.219467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] PATH_INFO: `/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:21.228844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] RBAC: Authorizing `identity:get_user(user_id=712ebe2bd9a64fd4b5168ac1dfac183c)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:21.230321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:21.232090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 118/227] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:21 2022] GET /identity/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:21.240961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:21.261646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:21.265172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:21.266601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:21.266924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:21.291508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:21.310596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:21.311558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:21.312402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 110/228] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:21.319304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:21.362696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:21.399772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9ba24-565a-42ab-9e5f-a8db16e6633f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:21.415014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:21.439127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:21.440331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:21.440738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:21.441130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:21.442429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:21.444013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:21.445850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 119/229] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:21 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:21.452442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:21.523914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47488883-7c7b-4bc5-94a0-d00dfa0666ac None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:21.558337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:21.559247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:21.559798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:21.560277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:21.561168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:21.561620 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:21.563152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:21.571509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 111/230] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:21.577063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:21.646344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c3c1c-28f0-4d42-90d4-10770ddbdc07 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:21.691048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:21.692273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:21.692726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:21.693187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973/users/712ebe2bd9a64fd4b5168ac1dfac183c/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:21.697570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] RBAC: Authorizing `identity:create_grant(project_id=47be6257c8c8422a87841e5a37f65973, user_id=712ebe2bd9a64fd4b5168ac1dfac183c, role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:21.699709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:21.731739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 120/231] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:21 2022] PUT /identity/v3/projects/47be6257c8c8422a87841e5a37f65973/users/712ebe2bd9a64fd4b5168ac1dfac183c/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:21.740397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:21.769922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966751c-5b33-4b86-9c2f-e806b7dd3985 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:21.823516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:21.824745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:21.825266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:21.825712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] PATH_INFO: `/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:21.828918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] RBAC: Authorizing `identity:get_user(user_id=712ebe2bd9a64fd4b5168ac1dfac183c)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:21.830455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:21.832633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 112/232] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:21 2022] GET /identity/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c => generated 283 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:21.837953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:21.931964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:22.005793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9702419-bc28-4a8d-aef8-3eb5a9f10ccd None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:22.044811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0074b0-e5f9-4006-8e0f-3782cc7e9636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:22.046040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0074b0-e5f9-4006-8e0f-3782cc7e9636 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:22.046523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0074b0-e5f9-4006-8e0f-3782cc7e9636 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:22.047032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0074b0-e5f9-4006-8e0f-3782cc7e9636 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:22.048496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0074b0-e5f9-4006-8e0f-3782cc7e9636 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:22.050163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0074b0-e5f9-4006-8e0f-3782cc7e9636 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:22.052531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 121/233] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:21 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:22.064048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:22.199938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d332050-f64f-4d2a-8a21-5c875d6e6c40 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:22.237889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8e103-a4e0-4905-a88b-23fcb50a24b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:22.238784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e103-a4e0-4905-a88b-23fcb50a24b4 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:22.239081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e103-a4e0-4905-a88b-23fcb50a24b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:22.239343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8e103-a4e0-4905-a88b-23fcb50a24b4 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973/users/712ebe2bd9a64fd4b5168ac1dfac183c/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:22.241349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8e103-a4e0-4905-a88b-23fcb50a24b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=47be6257c8c8422a87841e5a37f65973, user_id=712ebe2bd9a64fd4b5168ac1dfac183c, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:22.242799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8e103-a4e0-4905-a88b-23fcb50a24b4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:22.259014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 113/234] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:22 2022] PUT /identity/v3/projects/47be6257c8c8422a87841e5a37f65973/users/712ebe2bd9a64fd4b5168ac1dfac183c/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:23.025329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd70360-1dfb-4d26-ae9c-82a083f81ab4 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.025895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd70360-1dfb-4d26-ae9c-82a083f81ab4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.025895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd70360-1dfb-4d26-ae9c-82a083f81ab4 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.027515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 122/235] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:23.044818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f304-5d24-4511-8a57-93c84589dc8c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.045319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f304-5d24-4511-8a57-93c84589dc8c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.045407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f304-5d24-4511-8a57-93c84589dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.049749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b738ed5-0ccd-4458-9894-45559b91b6f3 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.050142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b738ed5-0ccd-4458-9894-45559b91b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.050528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b738ed5-0ccd-4458-9894-45559b91b6f3 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.052371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 123/236] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:23.057526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:23.058326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc1a9-176f-4871-b924-aa105a09dd06 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.058703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc1a9-176f-4871-b924-aa105a09dd06 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.059052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc1a9-176f-4871-b924-aa105a09dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.069660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-0a62f304-5d24-4511-8a57-93c84589dc8c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:23.071461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-0a62f304-5d24-4511-8a57-93c84589dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:23.079760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-68cfc1a9-176f-4871-b924-aa105a09dd06 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:23.081062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-68cfc1a9-176f-4871-b924-aa105a09dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:23.171607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 124/237] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:23.179561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 114/238] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:23.197475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc1a9-176f-4871-b924-aa105a09dd06 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.216997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f304-5d24-4511-8a57-93c84589dc8c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.246988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:23.311929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc1a9-176f-4871-b924-aa105a09dd06 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:23.348242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514fb14-0f0c-46e1-9e3f-565d7d744bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:23.349073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514fb14-0f0c-46e1-9e3f-565d7d744bcc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.349365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514fb14-0f0c-46e1-9e3f-565d7d744bcc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.349642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514fb14-0f0c-46e1-9e3f-565d7d744bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.350912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514fb14-0f0c-46e1-9e3f-565d7d744bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:23.352260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514fb14-0f0c-46e1-9e3f-565d7d744bcc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:23.353896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 125/239] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:23.355066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255c2b-3762-45a2-97c2-1176100c40a1 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.355348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255c2b-3762-45a2-97c2-1176100c40a1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.355622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2255c2b-3762-45a2-97c2-1176100c40a1 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.356987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 126/240] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:23.361843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2255c2b-3762-45a2-97c2-1176100c40a1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.369724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f304-5d24-4511-8a57-93c84589dc8c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:23.408967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee8abf-3e39-4004-8f66-fe47a41cc034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:23.409960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee8abf-3e39-4004-8f66-fe47a41cc034 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.410488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee8abf-3e39-4004-8f66-fe47a41cc034 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.410699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee8abf-3e39-4004-8f66-fe47a41cc034 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.413533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:23.423617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee8abf-3e39-4004-8f66-fe47a41cc034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-79ee8abf-3e39-4004-8f66-fe47a41cc034 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:23.431549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:23.432691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:23.433545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:23.439159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 115/241] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:23.439159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978070-b2a3-49cf-8ee4-34158b60fcb8 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.439159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978070-b2a3-49cf-8ee4-34158b60fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.439159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6978070-b2a3-49cf-8ee4-34158b60fcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.457430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d6978070-b2a3-49cf-8ee4-34158b60fcb8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:23.459223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d6978070-b2a3-49cf-8ee4-34158b60fcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:23.490018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2255c2b-3762-45a2-97c2-1176100c40a1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:23.502375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:23.537377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:23.538460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.538808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.539151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.539933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:23.541533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:23.542571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 116/242] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:23.543740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978070-b2a3-49cf-8ee4-34158b60fcb8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.579295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 127/243] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:23 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:23.580533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.689241 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6978070-b2a3-49cf-8ee4-34158b60fcb8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:23.722943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:23.734682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:23.736368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.736997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.737506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.738455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:23.738960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:23.740709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:23.751533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 117/244] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:23.757869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.785464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91324ab-b313-4cca-bcff-36e1585149b8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:23.830389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:23.831665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.832214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.832740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.846137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:23.855317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:23.856333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:23.857132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:23.859960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 128/245] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:23.866054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.871592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4c2b-0dce-48e4-88bd-a08644f15f16 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:23.894927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:23.908331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:23.909442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:23.909573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:23.909753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] PATH_INFO: `/v3/users/b51c1c21ca2244e19f14800c238765d3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:23.926512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] RBAC: Authorizing `identity:get_user(user_id=b51c1c21ca2244e19f14800c238765d3)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:23.928072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:23.929810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 118/246] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:23 2022] GET /identity/v3/users/b51c1c21ca2244e19f14800c238765d3 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:23.935284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:23.986238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:23.992357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945be301-0517-498f-bb91-3a10dde40a21 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.044644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.045402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.045590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.045835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.046718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:24.047022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:24.048842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:24.058772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 129/247] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:24.064759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:24.066743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de28f3-0dff-49d7-99d4-b8c1a5e6fcaf None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.104522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a16c02-2e72-4645-b158-14bb3ca48475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.105495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a16c02-2e72-4645-b158-14bb3ca48475 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.105991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a16c02-2e72-4645-b158-14bb3ca48475 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.106507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a16c02-2e72-4645-b158-14bb3ca48475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.161930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:24.178882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a16c02-2e72-4645-b158-14bb3ca48475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:24.179972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483af98-ccc0-446a-be81-408e0c285bc5 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.181084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a16c02-2e72-4645-b158-14bb3ca48475 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:24.218720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.219657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.219937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.220238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] PATH_INFO: `/v3/users/f55dec7957ca45cbad2f0c164a98ff0f` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.221463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] RBAC: Authorizing `identity:get_user(user_id=f55dec7957ca45cbad2f0c164a98ff0f)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:24.222917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:24.224696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 130/248] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:24 2022] GET /identity/v3/users/f55dec7957ca45cbad2f0c164a98ff0f => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:24.230901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:24.231668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:24.298841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 119/249] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:24.300223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016768c-9c82-40d9-bfa9-514a4bda92b8 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.300511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016768c-9c82-40d9-bfa9-514a4bda92b8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.300792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016768c-9c82-40d9-bfa9-514a4bda92b8 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.302322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 120/250] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:24.306959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016768c-9c82-40d9-bfa9-514a4bda92b8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:24.350599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157935c4-4ed9-438f-946c-fd8afb5b42bf None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.385966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec6931-f66d-4baf-8158-67abb292aada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.386755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec6931-f66d-4baf-8158-67abb292aada None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.387026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec6931-f66d-4baf-8158-67abb292aada None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.387295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec6931-f66d-4baf-8158-67abb292aada None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.388569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec6931-f66d-4baf-8158-67abb292aada None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:24.390096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec6931-f66d-4baf-8158-67abb292aada None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:24.396122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 131/251] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:24 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:24.397444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841c980-e5cb-4779-9b27-60c7d60ed06b None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.397729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841c980-e5cb-4779-9b27-60c7d60ed06b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.398001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841c980-e5cb-4779-9b27-60c7d60ed06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.414071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016768c-9c82-40d9-bfa9-514a4bda92b8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.415620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8841c980-e5cb-4779-9b27-60c7d60ed06b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:24.416829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8841c980-e5cb-4779-9b27-60c7d60ed06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:24.434536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:24.456798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.457849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.458223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.458587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.463305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:24.464903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:24.466967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 121/252] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:24 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:24.467777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:24.490301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 132/253] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:24.491492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841c980-e5cb-4779-9b27-60c7d60ed06b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:24.500377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:24.611580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f5f80-35cf-4fe3-979a-30993f271540 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.626196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841c980-e5cb-4779-9b27-60c7d60ed06b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.666284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.667102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.667382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.667662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973/users/f55dec7957ca45cbad2f0c164a98ff0f/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.671494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] RBAC: Authorizing `identity:create_grant(project_id=47be6257c8c8422a87841e5a37f65973, user_id=f55dec7957ca45cbad2f0c164a98ff0f, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:24.672981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:24.690439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.691554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.691805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.692134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.692975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] RBAC: Adding query filter params (role.id=ed5809ab31cb49e9b7122b1605a2e052, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=b51c1c21ca2244e19f14800c238765d3) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:24.693285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:24.693985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 122/254] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:24 2022] PUT /identity/v3/projects/47be6257c8c8422a87841e5a37f65973/users/f55dec7957ca45cbad2f0c164a98ff0f/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:24.694865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:24.699608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:24.717415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 133/255] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:24 2022] GET /identity/v3/role_assignments?user.id=b51c1c21ca2244e19f14800c238765d3&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=ed5809ab31cb49e9b7122b1605a2e052 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:24.835719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:24.873880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecedc6c5-1841-486c-ba1f-969ab6c3f10a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:24.920527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0f6f4-4b3f-4123-b10e-bb4d72e7f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:24.921268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f6f4-4b3f-4123-b10e-bb4d72e7f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.921545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f6f4-4b3f-4123-b10e-bb4d72e7f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.921815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0f6f4-4b3f-4123-b10e-bb4d72e7f1b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.942162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0f6f4-4b3f-4123-b10e-bb4d72e7f1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-1fb0f6f4-4b3f-4123-b10e-bb4d72e7f1b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:24.950374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:24.951334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:24.952259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 123/256] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:24.957609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:24.977610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:24.994529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9856b3-e045-4391-8237-eaae211cb404 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:24.994831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9856b3-e045-4391-8237-eaae211cb404 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:24.995083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9856b3-e045-4391-8237-eaae211cb404 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:24.996489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 124/257] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:25.015558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a965a7-4a99-4b86-bda3-66d9f700a5c0 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:25.015898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a965a7-4a99-4b86-bda3-66d9f700a5c0 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:25.016146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a965a7-4a99-4b86-bda3-66d9f700a5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:25.042146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-49a965a7-4a99-4b86-bda3-66d9f700a5c0 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:25.043328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-49a965a7-4a99-4b86-bda3-66d9f700a5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:25.137695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 125/258] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:25.146267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5dcab9-da90-4313-b4b5-3e94eeca9b50 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:25.174259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a965a7-4a99-4b86-bda3-66d9f700a5c0 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:25.202832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:25.203508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:25.203785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:25.204094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:25.205081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:25.205384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:25.207206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:25.223261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 134/259] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:25.228907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:25.314245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:25.406808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a965a7-4a99-4b86-bda3-66d9f700a5c0 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:25.411678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:25.468724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:25.469644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:25.469929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:25.470202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:25.485957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227f554-38d7-47e4-beab-165723298e7c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:25.497450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:25.505478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:25.506355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:25.507155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:25.511417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 126/260] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:25.517872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:25.565696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:25.566994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:25.567588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:25.568090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] PATH_INFO: `/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:25.569714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] RBAC: Authorizing `identity:get_user(user_id=712ebe2bd9a64fd4b5168ac1dfac183c)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:25.571832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:25.578404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 135/261] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:25 2022] GET /identity/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c => generated 283 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:25.583398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:25.690335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab59cb-ed4e-4f6d-b3f0-8df0233ea77f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:25.737064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:25.760969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27071203-6993-48a3-97ca-1c3937c80d9f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:25.768665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:25.769602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:25.770086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:25.770560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:25.771513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:25.772070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:25.773660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:25.791554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 127/262] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:25.797339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:25.821697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:25.822367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:25.822644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:25.822917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:25.851502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:25.961455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:25.963171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:26.039905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e1c87-4b5a-4a45-bc5b-6c344d8be292 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:26.118520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:26.119750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:26.120256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:26.120830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] PATH_INFO: `/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:26.124691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] RBAC: Authorizing `identity:get_user(user_id=712ebe2bd9a64fd4b5168ac1dfac183c)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:26.126366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:26.128284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 128/263] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:25 2022] GET /identity/v3/users/712ebe2bd9a64fd4b5168ac1dfac183c => generated 283 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:26.133786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:26.184681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 136/264] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 15:37:26.190141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:26.241854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:26.302890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:26.346153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b521672-5479-49db-aae4-b0ed58876b03 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:26.373500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aaad49-dcd6-4593-b618-ace2813f541f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:26.414753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:26.415408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:26.415692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:26.415968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:26.426085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:26.426997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:26.427274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:26.427540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:26.432937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:26.434485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:26.436662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 137/265] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:26 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:26.442320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:26.546669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:26.548170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:26.624957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db0e9b-b4ae-4694-bd9e-06f8575c1e5d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:26.653277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:26.653456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:26.695675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a86cb-76d5-46a1-a3d7-5b0f4be7bce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:26.696355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a86cb-76d5-46a1-a3d7-5b0f4be7bce3 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:26.696630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a86cb-76d5-46a1-a3d7-5b0f4be7bce3 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:26.696900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a86cb-76d5-46a1-a3d7-5b0f4be7bce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:26.697729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a86cb-76d5-46a1-a3d7-5b0f4be7bce3 None admin] RBAC: Adding query filter params (role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=712ebe2bd9a64fd4b5168ac1dfac183c) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:26.698034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a86cb-76d5-46a1-a3d7-5b0f4be7bce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:26.699493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a86cb-76d5-46a1-a3d7-5b0f4be7bce3 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:26.710868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 138/266] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:26 2022] GET /identity/v3/role_assignments?user.id=712ebe2bd9a64fd4b5168ac1dfac183c&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:26.713060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a48b7-7efb-4344-9ae5-d2294beb347e None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:26.713341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a48b7-7efb-4344-9ae5-d2294beb347e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:26.713617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a48b7-7efb-4344-9ae5-d2294beb347e None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:26.714861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 139/267] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:26.724086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217e32f-e85f-41a5-b542-62dc871550fb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:26.724433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217e32f-e85f-41a5-b542-62dc871550fb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:26.724638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217e32f-e85f-41a5-b542-62dc871550fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:26.753417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 129/268] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 15:37:26.754406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d217e32f-e85f-41a5-b542-62dc871550fb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:26.755606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d217e32f-e85f-41a5-b542-62dc871550fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:26.759382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:26.930204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 140/269] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:26.955296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d710-680f-4eea-a7f0-da3d1010661e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:26.995635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:26.996527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:26.996867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:26.997135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.000885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:27.002358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:27.004273 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 130/270] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:26 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:27.010084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217e32f-e85f-41a5-b542-62dc871550fb None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.022731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.072137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:27.188627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217e32f-e85f-41a5-b542-62dc871550fb None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:27.189998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa269d21-1319-468b-b813-13e50b088079 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:27.200881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:27.232181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:27.233291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.233796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.234314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.235352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=712ebe2bd9a64fd4b5168ac1dfac183c) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:27.235891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:27.237627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:27.244555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 131/271] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/role_assignments?user.id=712ebe2bd9a64fd4b5168ac1dfac183c&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:27.246293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fa25c-dc4c-48cc-86c1-6760e68402ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:27.247167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fa25c-dc4c-48cc-86c1-6760e68402ed None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.247444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fa25c-dc4c-48cc-86c1-6760e68402ed None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.247714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fa25c-dc4c-48cc-86c1-6760e68402ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.270022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fa25c-dc4c-48cc-86c1-6760e68402ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-fb4fa25c-dc4c-48cc-86c1-6760e68402ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:27.282674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:27.283632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:27.284503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 141/272] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:27.290154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.320041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b053eb-6a1d-4ce6-bffb-c804261b93ec None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.320678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b053eb-6a1d-4ce6-bffb-c804261b93ec None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.321162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b053eb-6a1d-4ce6-bffb-c804261b93ec None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.322759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 142/273] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:27.333837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb59f4a-a32a-4d63-93a7-75d5d3c85ef4 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.334444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb59f4a-a32a-4d63-93a7-75d5d3c85ef4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.334905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb59f4a-a32a-4d63-93a7-75d5d3c85ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.368091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1bb59f4a-a32a-4d63-93a7-75d5d3c85ef4 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:27.368091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1bb59f4a-a32a-4d63-93a7-75d5d3c85ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:27.394662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:27.412890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b6685-e588-40e6-b17c-02bb70b7b496 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:27.451273 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:27.451973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.452301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.452577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.453341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:27.453765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:27.455175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:27.465231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 132/274] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:27.471899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.507136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 143/275] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:27.529727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2177ea0-5e2a-4503-ad4f-fea43bfd424a None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.530349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2177ea0-5e2a-4503-ad4f-fea43bfd424a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.530626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2177ea0-5e2a-4503-ad4f-fea43bfd424a None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.532175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 144/276] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:27.537519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fef31-3bbf-4ecf-a623-54bf7d7547ac None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.538044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fef31-3bbf-4ecf-a623-54bf7d7547ac None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.538508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fef31-3bbf-4ecf-a623-54bf7d7547ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.560391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-684fef31-3bbf-4ecf-a623-54bf7d7547ac None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:27.561811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-684fef31-3bbf-4ecf-a623-54bf7d7547ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:27.600069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591600c-c272-4f6b-91c9-a62ea26a24c5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:27.636410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dfe76-9c30-4399-99c1-3633d22b51ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:27.637346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dfe76-9c30-4399-99c1-3633d22b51ef None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.637627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dfe76-9c30-4399-99c1-3633d22b51ef None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.637895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dfe76-9c30-4399-99c1-3633d22b51ef None admin] PATH_INFO: `/v3/users/bfa389967a0947219f606c2641f308d4` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.642887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 145/277] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:27.643741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fef31-3bbf-4ecf-a623-54bf7d7547ac None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.656173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dfe76-9c30-4399-99c1-3633d22b51ef None admin] RBAC: Authorizing `identity:get_user(user_id=bfa389967a0947219f606c2641f308d4)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:27.657646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dfe76-9c30-4399-99c1-3633d22b51ef None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:27.659371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 133/278] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/users/bfa389967a0947219f606c2641f308d4 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:27.665206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d17a-2093-41e8-99ef-7044cb01c16a None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.665498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d17a-2093-41e8-99ef-7044cb01c16a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.665772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9d17a-2093-41e8-99ef-7044cb01c16a None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.667368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 134/279] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:27.668196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d17a-2093-41e8-99ef-7044cb01c16a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.675414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:27.677212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:27.747530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fef31-3bbf-4ecf-a623-54bf7d7547ac None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:27.769938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9d17a-2093-41e8-99ef-7044cb01c16a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:27.781135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:27.781921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.782195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.782462 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.783607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:27.785097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:27.786689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 146/280] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:27.787453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.803719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbc2b5-8d59-45d4-9fbd-1a87e560597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:27.804554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbc2b5-8d59-45d4-9fbd-1a87e560597e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.804833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbc2b5-8d59-45d4-9fbd-1a87e560597e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.805099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbc2b5-8d59-45d4-9fbd-1a87e560597e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.815224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbc2b5-8d59-45d4-9fbd-1a87e560597e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-78bbc2b5-8d59-45d4-9fbd-1a87e560597e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:27.822894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:27.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 135/281] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:27.829763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061bc2e-c863-4fce-894c-b07d90feddcc None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.830506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061bc2e-c863-4fce-894c-b07d90feddcc None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.830506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061bc2e-c863-4fce-894c-b07d90feddcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.844062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3061bc2e-c863-4fce-894c-b07d90feddcc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:27.845282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3061bc2e-c863-4fce-894c-b07d90feddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:27.882546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:27.889345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2b706e-8a57-4067-beb6-ffb2b21e4580 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:27.911486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:27.919591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 136/282] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:27.920445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061bc2e-c863-4fce-894c-b07d90feddcc None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:27.924308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:27.924971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:27.925256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:27.925531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:27.926372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, user.id=bfa389967a0947219f606c2641f308d4, scope.system=all) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:27.926668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:27.928157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:27.933306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 147/283] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/role_assignments?user.id=bfa389967a0947219f606c2641f308d4&scope.system=all&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:27.934361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.049133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061bc2e-c863-4fce-894c-b07d90feddcc None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.058356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4d31c-8e2a-4fcd-b493-1d4a8f71f1f3 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.085878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.086557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.086838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.087113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.088030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:28.089345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:28.100385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.101245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.101527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.101800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.102551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:28.102856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:28.104269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:28.114398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 148/284] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:28.115227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.117386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 137/285] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:27 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:28.120108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.160570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:28.194226 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:28.272455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50273d00-3448-4dbc-b5f1-c7a0906fdbda None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.273281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bed671-9a18-4681-b0b9-a736b43f9e28 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.318470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.318654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.318886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.319120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.330738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.331944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.332571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.332807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:28.333282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] PATH_INFO: `/v3/users/f55dec7957ca45cbad2f0c164a98ff0f` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.334710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] RBAC: Authorizing `identity:get_user(user_id=f55dec7957ca45cbad2f0c164a98ff0f)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:28.336403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:28.338333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 138/286] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/users/f55dec7957ca45cbad2f0c164a98ff0f => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:28.341607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:28.342488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:28.343300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 149/287] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:28.344483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.350807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.456241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:28.471357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24859c29-b2a1-4805-ad26-3d0e1ae933a7 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.478466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0360f-178c-4efe-8297-4338a8b04ce1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.494817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:28.511667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.512440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.512785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.512992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.521000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.521679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.521966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.522240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.522987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] RBAC: Adding query filter params (name=reader) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:28.523293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:28.525139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:28.536245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 150/288] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:28.543811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.633128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:28.634464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:28.676939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff2163-85fd-4355-aef2-56ccdab3686d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.725612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.726914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.727499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.727969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] PATH_INFO: `/v3/users/b51c1c21ca2244e19f14800c238765d3` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.732779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] RBAC: Authorizing `identity:get_user(user_id=b51c1c21ca2244e19f14800c238765d3)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:28.734471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:28.736556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 151/289] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/users/b51c1c21ca2244e19f14800c238765d3 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:28.742105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.771968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 139/290] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 15:37:28.772664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:28.776354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:28.804407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:28.881970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b566bed-729d-46c4-9bee-d64e2b3c9cad None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:28.919216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:28.920015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:28.920374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:28.920649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:28.921846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:28.923715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:28.925774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 140/291] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:28.932197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:29.014678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feabfc-1781-40d3-b9f6-5c56a03e94ba None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:29.024374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:29.038027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3fec09-bf4f-4da3-8e88-fdb12a8660a1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:29.074650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:29.075817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:29.076135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:29.076646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:29.077586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=f55dec7957ca45cbad2f0c164a98ff0f) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:29.077996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:29.079649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:29.086587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 141/292] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/role_assignments?user.id=f55dec7957ca45cbad2f0c164a98ff0f&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:29.132138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c59d86-e517-4f85-a1cd-9837b2fc20c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:29.133042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59d86-e517-4f85-a1cd-9837b2fc20c7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:29.133324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59d86-e517-4f85-a1cd-9837b2fc20c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:29.133600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c59d86-e517-4f85-a1cd-9837b2fc20c7 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:29.140730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59d86-e517-4f85-a1cd-9837b2fc20c7 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:29.142241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c59d86-e517-4f85-a1cd-9837b2fc20c7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:29.144136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 152/293] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:28 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:29.150263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:29.310464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183197b-04ae-4a82-a6fb-5bd75c32ce11 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:29.347003 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1158cb-944f-4213-99e0-696bb3b6789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:29.347892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1158cb-944f-4213-99e0-696bb3b6789e None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:29.348196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1158cb-944f-4213-99e0-696bb3b6789e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:29.348467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1158cb-944f-4213-99e0-696bb3b6789e None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973/users/b51c1c21ca2244e19f14800c238765d3/roles/ed5809ab31cb49e9b7122b1605a2e052` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:29.350526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1158cb-944f-4213-99e0-696bb3b6789e None admin] RBAC: Authorizing `identity:create_grant(project_id=47be6257c8c8422a87841e5a37f65973, user_id=b51c1c21ca2244e19f14800c238765d3, role_id=ed5809ab31cb49e9b7122b1605a2e052)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:29.352045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1158cb-944f-4213-99e0-696bb3b6789e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:29.366708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 142/294] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:37:29 2022] PUT /identity/v3/projects/47be6257c8c8422a87841e5a37f65973/users/b51c1c21ca2244e19f14800c238765d3/roles/ed5809ab31cb49e9b7122b1605a2e052 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:30.257349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13cab6-b4d8-4626-afa0-dbc1f7f31e12 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.257897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13cab6-b4d8-4626-afa0-dbc1f7f31e12 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.257897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13cab6-b4d8-4626-afa0-dbc1f7f31e12 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.259543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 153/295] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:30.266144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271398f3-5ff9-4a6c-88c7-737eb1eccb47 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.266144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271398f3-5ff9-4a6c-88c7-737eb1eccb47 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.266144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271398f3-5ff9-4a6c-88c7-737eb1eccb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.285390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-271398f3-5ff9-4a6c-88c7-737eb1eccb47 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:30.286601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-271398f3-5ff9-4a6c-88c7-737eb1eccb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:30.294839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:30.357286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 143/296] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:30.379775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13cab6-b4d8-4626-afa0-dbc1f7f31e12 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:30.447605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f01060-443a-4c70-a1fb-f9066246fbb1 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.447894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f01060-443a-4c70-a1fb-f9066246fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.448412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f01060-443a-4c70-a1fb-f9066246fbb1 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.450315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 144/297] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:30.455408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868e45-a8ed-4e04-96d0-afbaf76b957b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.455922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868e45-a8ed-4e04-96d0-afbaf76b957b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.456388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee868e45-a8ed-4e04-96d0-afbaf76b957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.472989 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ee868e45-a8ed-4e04-96d0-afbaf76b957b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:30.473294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:30.474672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ee868e45-a8ed-4e04-96d0-afbaf76b957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:30.495552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13cab6-b4d8-4626-afa0-dbc1f7f31e12 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:30.537402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:30.538116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.538391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.538659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.548983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:30.553274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 145/298] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:30.554830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a3fe7-6d76-4549-a9bf-9eb1b4edf9bf None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.555117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a3fe7-6d76-4549-a9bf-9eb1b4edf9bf None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.555392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a3fe7-6d76-4549-a9bf-9eb1b4edf9bf None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.556690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 146/299] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:30.564142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:30.565053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:30.565850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:30.566445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf47931-e0ef-4adb-9867-94c49d75f8a4 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.566445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf47931-e0ef-4adb-9867-94c49d75f8a4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.566445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf47931-e0ef-4adb-9867-94c49d75f8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.568355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 154/300] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:30.573842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:30.584329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-2cf47931-e0ef-4adb-9867-94c49d75f8a4 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:30.585513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2cf47931-e0ef-4adb-9867-94c49d75f8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:30.654542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 147/301] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:30.656677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf47931-e0ef-4adb-9867-94c49d75f8a4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:30.659045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:30.687420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15212434-1e98-4922-9ad7-8e4bdb731487 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:30.719240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:30.719905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.720208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.720479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.721223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:30.721526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:30.722898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:30.731017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 155/302] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:30.731802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:30.734454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:30.771802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf47931-e0ef-4adb-9867-94c49d75f8a4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:30.807245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:30.808105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.808386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.808655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.809907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:30.811331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:30.813157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 148/303] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:30.818166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:30.835652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620245f-f8d4-4cac-b19a-39a7b1e42119 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:30.870084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:30.870890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.871174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.871453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.881929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:30.889977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:30.890873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:30.891696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 156/304] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:30.892397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:30.893988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:30.928333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03933de3-2385-4ee3-9514-bbc913101247 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:30.959143 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:30.964072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:30.965023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:30.965304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:30.965575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] PATH_INFO: `/v3/users/bfa389967a0947219f606c2641f308d4` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:30.966763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] RBAC: Authorizing `identity:get_user(user_id=bfa389967a0947219f606c2641f308d4)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:30.968422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:30.970050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 149/305] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:30 2022] GET /identity/v3/users/bfa389967a0947219f606c2641f308d4 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:30.970701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:31.006115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac2e79-fad9-45f5-8285-966a4dd13392 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:31.042376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:31.043044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.043327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.043654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] PATH_INFO: `/v3/groups` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.044387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:31.045847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:31.062748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 157/306] 10.222.0.50 () {60 vars in 1134 bytes} [Thu Jun 30 15:37:30 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:31.063297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:31.089238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82101db0-439d-4429-9e4c-578ef9b72868 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:31.128416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:31.129494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.130255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.130684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.131693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:31.132212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:31.134399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:31.143373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:31.152454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 150/307] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:31.155279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:31.207833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7751375-99d6-47e3-8ef0-0d198e2af4c4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:31.237457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:31.252830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:31.253862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.254145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.254422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] PATH_INFO: `/v3/system/users/bfa389967a0947219f606c2641f308d4/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.256790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bfa389967a0947219f606c2641f308d4, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:31.258214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:31.270881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 158/308] 10.222.0.50 () {58 vars in 1334 bytes} [Thu Jun 30 15:37:31 2022] PUT /identity/v3/system/users/bfa389967a0947219f606c2641f308d4/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:31.299567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873dbad0-db9e-4101-b34a-4a53406479d0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:31.340923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278cb7f-ec8c-476b-b127-e2f66497980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:31.342290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278cb7f-ec8c-476b-b127-e2f66497980b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.342762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278cb7f-ec8c-476b-b127-e2f66497980b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.343341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278cb7f-ec8c-476b-b127-e2f66497980b None admin] PATH_INFO: `/v3/users/7523e7cebce348349a764ae00ae50894` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.360569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278cb7f-ec8c-476b-b127-e2f66497980b None admin] RBAC: Authorizing `identity:get_user(user_id=7523e7cebce348349a764ae00ae50894)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:31.362957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278cb7f-ec8c-476b-b127-e2f66497980b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:31.368657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 151/309] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:31 2022] GET /identity/v3/users/7523e7cebce348349a764ae00ae50894 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:31.372092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:31.438912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644759b1-5d8d-4b43-899a-68fa6ad18275 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.439242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644759b1-5d8d-4b43-899a-68fa6ad18275 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.439468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644759b1-5d8d-4b43-899a-68fa6ad18275 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.441466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 152/310] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:31.448434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09592b-2bef-4df6-9aa7-f98ed9c6df65 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.448725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09592b-2bef-4df6-9aa7-f98ed9c6df65 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.449343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09592b-2bef-4df6-9aa7-f98ed9c6df65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.466255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1e09592b-2bef-4df6-9aa7-f98ed9c6df65 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:31.467458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1e09592b-2bef-4df6-9aa7-f98ed9c6df65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:31.498589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e6f8f-1dee-441a-a27f-501883649adf None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:31.531367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:31.544813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:31.545470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.545748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.546144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.546887 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 153/311] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:31.547084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] RBAC: Adding query filter params (role.id=ed5809ab31cb49e9b7122b1605a2e052, user.id=7523e7cebce348349a764ae00ae50894, scope.system=all) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:31.547425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:31.548959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:31.554145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 159/312] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Jun 30 15:37:31 2022] GET /identity/v3/role_assignments?user.id=7523e7cebce348349a764ae00ae50894&scope.system=all&role.id=ed5809ab31cb49e9b7122b1605a2e052 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:31.571650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09592b-2bef-4df6-9aa7-f98ed9c6df65 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:31.676460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09592b-2bef-4df6-9aa7-f98ed9c6df65 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:31.711715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:31.712555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.712837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.713106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.724800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:31.732891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:31.733833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:31.734666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 154/313] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:31.741530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:31.796365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:31.849328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538efa66-5c36-4baa-a082-e0a82a03ef9d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:31.883607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:31.884311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:31.884586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:31.884857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:31.885926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] RBAC: Adding query filter params (name=reader) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:31.886226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:31.887672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:31.896497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 160/314] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:31.902118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:31.958447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:32.010149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4a26b-8254-49c9-b9be-2649f2806226 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:32.045460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:32.046448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:32.046728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:32.047057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] PATH_INFO: `/v3/users/b51c1c21ca2244e19f14800c238765d3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:32.048284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] RBAC: Authorizing `identity:get_user(user_id=b51c1c21ca2244e19f14800c238765d3)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:32.049741 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:32.051717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 155/315] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:31 2022] GET /identity/v3/users/b51c1c21ca2244e19f14800c238765d3 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:32.057099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:32.162112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7361e-d183-4801-b783-9f620416cbcb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:32.177116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:32.195786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:32.196885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:32.197368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:32.197974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:32.266529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:32.268019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:32.371811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 161/316] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:32.377539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:32.482354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c67067-7e52-47b7-a203-6b0f4f1fefbb None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:32.499014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:32.517785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2e3d7-4367-4d3b-80c8-6fbaea32e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:32.518578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e3d7-4367-4d3b-80c8-6fbaea32e51c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:32.518859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e3d7-4367-4d3b-80c8-6fbaea32e51c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:32.519132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2e3d7-4367-4d3b-80c8-6fbaea32e51c None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:32.520348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2e3d7-4367-4d3b-80c8-6fbaea32e51c None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:32.521803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2e3d7-4367-4d3b-80c8-6fbaea32e51c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:32.523795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 156/317] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:32 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:32.529699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:32.543954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:32.632310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad714ee-2510-48c0-8960-eac4b89affd0 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:32.666406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ce7dc-9742-4eff-b54f-e0294aa64e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:32.667104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce7dc-9742-4eff-b54f-e0294aa64e64 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:32.667384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce7dc-9742-4eff-b54f-e0294aa64e64 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:32.667657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ce7dc-9742-4eff-b54f-e0294aa64e64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:32.668560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ce7dc-9742-4eff-b54f-e0294aa64e64 None admin] RBAC: Adding query filter params (role.id=ed5809ab31cb49e9b7122b1605a2e052, scope.project.id=47be6257c8c8422a87841e5a37f65973, user.id=b51c1c21ca2244e19f14800c238765d3) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:32.668868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ce7dc-9742-4eff-b54f-e0294aa64e64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:32.670318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ce7dc-9742-4eff-b54f-e0294aa64e64 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:32.676865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 162/318] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:32 2022] GET /identity/v3/role_assignments?user.id=b51c1c21ca2244e19f14800c238765d3&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=ed5809ab31cb49e9b7122b1605a2e052 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:33.217455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e6e7-b1da-42a8-9fac-3d1071e34a58 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.217708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e6e7-b1da-42a8-9fac-3d1071e34a58 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.217986 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9e6e7-b1da-42a8-9fac-3d1071e34a58 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.219541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 157/319] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:33.224703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e5452-70ab-47b6-829e-ead4c24feb48 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.225002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e5452-70ab-47b6-829e-ead4c24feb48 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.225232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e5452-70ab-47b6-829e-ead4c24feb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.242497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-003e5452-70ab-47b6-829e-ead4c24feb48 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:33.243688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-003e5452-70ab-47b6-829e-ead4c24feb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:33.318697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 163/320] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:33.341050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9e6e7-b1da-42a8-9fac-3d1071e34a58 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:33.402457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd829e-0e11-4c41-9046-5c6b28ce6462 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.402818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd829e-0e11-4c41-9046-5c6b28ce6462 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.403104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd829e-0e11-4c41-9046-5c6b28ce6462 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.404627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 164/321] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:33.409324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412502-524b-480e-aad2-bd051f87f555 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.409611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412502-524b-480e-aad2-bd051f87f555 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.409890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f412502-524b-480e-aad2-bd051f87f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.420703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:33.426739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7f412502-524b-480e-aad2-bd051f87f555 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:33.428013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7f412502-524b-480e-aad2-bd051f87f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:33.447896 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9e6e7-b1da-42a8-9fac-3d1071e34a58 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:33.485038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:33.485750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.486025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.486309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.487456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:33.488941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:33.490578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 158/322] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:33.496152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:33.496634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 165/323] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:33.519674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412502-524b-480e-aad2-bd051f87f555 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:33.555338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:33.607124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e1706-1d40-4dec-8699-ad406d519509 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:33.618489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f412502-524b-480e-aad2-bd051f87f555 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:33.643652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e8ff6-3d55-4b32-bbf8-52ec3d0239ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:33.644346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e8ff6-3d55-4b32-bbf8-52ec3d0239ce None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.644628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e8ff6-3d55-4b32-bbf8-52ec3d0239ce None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.644898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e8ff6-3d55-4b32-bbf8-52ec3d0239ce None admin] PATH_INFO: `/v3/groups` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.645589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e8ff6-3d55-4b32-bbf8-52ec3d0239ce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:33.647152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7e8ff6-3d55-4b32-bbf8-52ec3d0239ce None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:33.653951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:33.654752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.655142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.655420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.658706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:33.663775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 159/324] 10.222.0.50 () {60 vars in 1134 bytes} [Thu Jun 30 15:37:33 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:37:33.666857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d3ddf-c7e9-4aa3-bff1-626017db30be None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.667144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d3ddf-c7e9-4aa3-bff1-626017db30be None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.667422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d3ddf-c7e9-4aa3-bff1-626017db30be None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.668961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 160/325] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:33.673062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:33.674616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b29c60-0a58-400b-8f8b-7e7b5a0ece21 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.674912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b29c60-0a58-400b-8f8b-7e7b5a0ece21 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.675195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b29c60-0a58-400b-8f8b-7e7b5a0ece21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:33.682631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:33.683607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:33.684554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 166/326] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:33.691622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:33.694374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c4b29c60-0a58-400b-8f8b-7e7b5a0ece21 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:33.695650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c4b29c60-0a58-400b-8f8b-7e7b5a0ece21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:33.774918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 161/327] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:33.791940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b29c60-0a58-400b-8f8b-7e7b5a0ece21 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:33.808909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a4302-4080-4e32-b3c8-037e176ce7f6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:33.841022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:33.853881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:33.854564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.854841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.855111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.855902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:33.856232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:33.857621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:33.868395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 167/328] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:33.874178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:33.919181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b29c60-0a58-400b-8f8b-7e7b5a0ece21 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:33.954650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:33.966133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:33.966313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:33.967026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:33.967026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:33.977902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:33.986606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:33.987549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:33.988440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 162/329] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:33.993706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:34.010559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfd536-4b3e-4674-aada-c81908c9548f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:34.054923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:34.055950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:34.056288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:34.056678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] PATH_INFO: `/v3/users/bfa389967a0947219f606c2641f308d4` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:34.057889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=bfa389967a0947219f606c2641f308d4)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:34.059451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:34.061310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 168/330] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:33 2022] GET /identity/v3/users/bfa389967a0947219f606c2641f308d4 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:34.066827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:34.136128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55b9f-263c-43d0-a7df-ee29cc2e5c21 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:34.137962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:34.182645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:34.183331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:34.183618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:34.184239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:34.185001 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:34.185544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:34.187488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:34.194175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04195ea9-7c06-4ae1-8092-0680d7681ff6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:34.197753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 163/331] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:34.207729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:34.221746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:34.231599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:34.232306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:34.232589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:34.232866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:34.233764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] RBAC: Adding query filter params (role.id=fd1cdca617c24c089cbe498be4cd05f8, user.id=bfa389967a0947219f606c2641f308d4, scope.system=all) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:34.234074 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:34.235583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:34.250051 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 169/332] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Jun 30 15:37:34 2022] GET /identity/v3/role_assignments?user.id=bfa389967a0947219f606c2641f308d4&scope.system=all&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:34.325042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911912c-93d6-412a-a0cf-3f86526beef6 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:34.362560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c56a24-253e-4134-a3ad-4feb744b4141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:34.363459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c56a24-253e-4134-a3ad-4feb744b4141 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:34.363744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c56a24-253e-4134-a3ad-4feb744b4141 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:34.364048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c56a24-253e-4134-a3ad-4feb744b4141 None admin] PATH_INFO: `/v3/users/7523e7cebce348349a764ae00ae50894` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:34.365381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c56a24-253e-4134-a3ad-4feb744b4141 None admin] RBAC: Authorizing `identity:get_user(user_id=7523e7cebce348349a764ae00ae50894)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:34.366862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c56a24-253e-4134-a3ad-4feb744b4141 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:34.368887 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 164/333] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:34 2022] GET /identity/v3/users/7523e7cebce348349a764ae00ae50894 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:34.374759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:34.516874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a2a78-2d1b-4854-a876-3b7d66cc6ce6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:34.554808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694cf35-d4f2-4553-b8d1-ad5cd99ce8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:34.555679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694cf35-d4f2-4553-b8d1-ad5cd99ce8cd None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:34.556021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694cf35-d4f2-4553-b8d1-ad5cd99ce8cd None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:34.556297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694cf35-d4f2-4553-b8d1-ad5cd99ce8cd None admin] PATH_INFO: `/v3/system/users/7523e7cebce348349a764ae00ae50894/roles/ed5809ab31cb49e9b7122b1605a2e052` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:34.557798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694cf35-d4f2-4553-b8d1-ad5cd99ce8cd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7523e7cebce348349a764ae00ae50894, role_id=ed5809ab31cb49e9b7122b1605a2e052)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:34.559239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694cf35-d4f2-4553-b8d1-ad5cd99ce8cd None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:34.575243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 170/334] 10.222.0.50 () {58 vars in 1334 bytes} [Thu Jun 30 15:37:34 2022] PUT /identity/v3/system/users/7523e7cebce348349a764ae00ae50894/roles/ed5809ab31cb49e9b7122b1605a2e052 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:35.916354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110f3da-55b0-4c44-937a-741dd2934717 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:35.916896 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110f3da-55b0-4c44-937a-741dd2934717 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:35.916896 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110f3da-55b0-4c44-937a-741dd2934717 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:35.918622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 165/335] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:35.924083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f982-8872-42f6-8bf8-a5b5c8475e1b None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:35.924495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f982-8872-42f6-8bf8-a5b5c8475e1b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:35.924671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34f982-8872-42f6-8bf8-a5b5c8475e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:35.942168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-cf34f982-8872-42f6-8bf8-a5b5c8475e1b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:35.943365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-cf34f982-8872-42f6-8bf8-a5b5c8475e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:35.997558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:36.024555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 171/336] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:36.060299 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110f3da-55b0-4c44-937a-741dd2934717 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:36.103710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:36.198783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110f3da-55b0-4c44-937a-741dd2934717 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:36.238269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:36.239067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:36.239344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:36.239614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:36.260962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:36.270463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:36.271621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:36.272653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 166/337] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:36.282044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f982-8872-42f6-8bf8-a5b5c8475e1b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:36.426493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34f982-8872-42f6-8bf8-a5b5c8475e1b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:36.485992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:36.486678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:36.486960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:36.487235 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:36.488009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:36.488318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:36.489707 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:36.504467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 172/338] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:36.509993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:36.651386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:36.662195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355dd40-6703-433e-8ac4-906e99a7080c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:36.710769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a925c5-675b-4ecd-932f-fb9b25050613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:36.711812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a925c5-675b-4ecd-932f-fb9b25050613 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:36.712356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a925c5-675b-4ecd-932f-fb9b25050613 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:36.712824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a925c5-675b-4ecd-932f-fb9b25050613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:36.799576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a925c5-675b-4ecd-932f-fb9b25050613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:36.801564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a925c5-675b-4ecd-932f-fb9b25050613 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:36.905733 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:36.918966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 167/339] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:36.924765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:36.933261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dec6f-11b4-4c6c-8e3d-948de5eecc96 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:36.933823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dec6f-11b4-4c6c-8e3d-948de5eecc96 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:36.934372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dec6f-11b4-4c6c-8e3d-948de5eecc96 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:36.936275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 168/340] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:36.951851 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c8d5-eb5a-480e-91d3-8f77fd8637e9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:36.952937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c8d5-eb5a-480e-91d3-8f77fd8637e9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:36.953423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c8d5-eb5a-480e-91d3-8f77fd8637e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:36.983253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d602c8d5-eb5a-480e-91d3-8f77fd8637e9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:36.985344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d602c8d5-eb5a-480e-91d3-8f77fd8637e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:37.015522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:37.076442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 169/341] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:37.078154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d5e4-a605-4491-9c23-129cf9e4e6ee None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.078651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d5e4-a605-4491-9c23-129cf9e4e6ee None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.079131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7d5e4-a605-4491-9c23-129cf9e4e6ee None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.080717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 170/342] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:37.086076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a25d1-b4e9-4f8c-a8ba-3d0d3424ee9e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.086603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a25d1-b4e9-4f8c-a8ba-3d0d3424ee9e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.087075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a25d1-b4e9-4f8c-a8ba-3d0d3424ee9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.093197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22132266-cefe-47c8-b86a-aa2cd2515f62 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.112970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-765a25d1-b4e9-4f8c-a8ba-3d0d3424ee9e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:37.114484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-765a25d1-b4e9-4f8c-a8ba-3d0d3424ee9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:37.132377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.133319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.133823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.133930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.135393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:37.140306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:37.142449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 173/343] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:36 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:37.147801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.193684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 171/344] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:37.194979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a25d1-b4e9-4f8c-a8ba-3d0d3424ee9e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.250388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:37.259857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d5be9e-bbb1-4774-971b-498929b208d9 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.270863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:37.309378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.310273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.310563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.311067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.317651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:37.319080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:37.320832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a25d1-b4e9-4f8c-a8ba-3d0d3424ee9e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.329607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 174/345] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:37.330389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.358614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.360685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.361169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.361728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.374533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:37.382943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:37.383827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:37.384680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:37.387858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 172/346] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:37.388869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.463297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053cde0-6ae9-48e8-9615-4ceacb3ba37a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.497860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.498683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.498966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.499242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.501341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35cedb-538c-4c87-a9d4-37c074146af3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.510616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:37.516869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:37.519206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:37.520224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 175/347] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:37.521457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.544956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be81aabf-38d1-4bde-a03b-979e262311e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.545975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81aabf-38d1-4bde-a03b-979e262311e6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.546399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81aabf-38d1-4bde-a03b-979e262311e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.546815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be81aabf-38d1-4bde-a03b-979e262311e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.547783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81aabf-38d1-4bde-a03b-979e262311e6 None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=3418ad75d020471a91b2b3cffb565408) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:37.548264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81aabf-38d1-4bde-a03b-979e262311e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:37.550055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be81aabf-38d1-4bde-a03b-979e262311e6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:37.555675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:37.557710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 173/348] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:37.559786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40408-42c6-45fd-930e-b333e6919e34 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.560285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40408-42c6-45fd-930e-b333e6919e34 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.560705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40408-42c6-45fd-930e-b333e6919e34 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.562633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 174/349] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:37.563605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40408-42c6-45fd-930e-b333e6919e34 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.632979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562001ce-b516-463f-ab88-3af4126f59f6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.671287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c850d-f0c2-406c-98da-16eb61a36c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.671974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c850d-f0c2-406c-98da-16eb61a36c88 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.672287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c850d-f0c2-406c-98da-16eb61a36c88 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.672601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c850d-f0c2-406c-98da-16eb61a36c88 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.673412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c850d-f0c2-406c-98da-16eb61a36c88 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:37.673961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c850d-f0c2-406c-98da-16eb61a36c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:37.675691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76c850d-f0c2-406c-98da-16eb61a36c88 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:37.683478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40408-42c6-45fd-930e-b333e6919e34 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.685852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 176/350] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:37.687219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051680-0e0f-408e-8058-cad277d86900 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.687512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051680-0e0f-408e-8058-cad277d86900 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.687793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051680-0e0f-408e-8058-cad277d86900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.710686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-15051680-0e0f-408e-8058-cad277d86900 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:37.712239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-15051680-0e0f-408e-8058-cad277d86900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:37.723962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.724956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.725377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.725831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.726727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:37.727263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:37.728915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:37.738626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 175/351] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:37.739856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.785328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 177/352] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:37.786261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15051680-0e0f-408e-8058-cad277d86900 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:37.786791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:37.812747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:37.871188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f26735-486e-41c5-9188-c4d05ee638e6 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.893724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15051680-0e0f-408e-8058-cad277d86900 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:37.911788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.912796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.913218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.913632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.930807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:37.931523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:37.931809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:37.932156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:37.997030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.000964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.011229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.012800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.038068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.048864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.117592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 176/353] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:38.118029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 178/354] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:38.118706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.124326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.223964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111c6dd-290d-475c-b8f4-c756456ab32a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.234739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a20587-4ea0-4d26-8d2e-c1ae81171652 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.249113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.260935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.262827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.263104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.263375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.275053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:38.283189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:38.284127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:38.284921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 179/355] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:38.285587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.289193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.290256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.290762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.291174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.292590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.294224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.296622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 177/356] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.297728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.307893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.392944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8647c-2853-4c23-9243-e61ab3d3512c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb31e26-b191-4c24-b136-d6c81170bb6e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.429015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.429547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.429827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.430099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.431301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.432906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.434666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 180/357] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.435428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.447566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.448518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.449033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.449495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.450380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:38.450843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.452487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.461781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 178/358] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.462903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.499529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.535833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.554513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88779e1-6880-429a-ad93-008c39dfb927 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.573440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bda9f-34a7-4dbf-92dd-a0feb9745363 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.591095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b315350-4f4e-4bfe-92c2-0e8c1cfbbb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.592381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b315350-4f4e-4bfe-92c2-0e8c1cfbbb53 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.592838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b315350-4f4e-4bfe-92c2-0e8c1cfbbb53 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.593306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b315350-4f4e-4bfe-92c2-0e8c1cfbbb53 None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.594850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b315350-4f4e-4bfe-92c2-0e8c1cfbbb53 None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.596561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b315350-4f4e-4bfe-92c2-0e8c1cfbbb53 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.598586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 181/359] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.600117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb9f47-faae-4354-85da-bfd4e88bdbae None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.600825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb9f47-faae-4354-85da-bfd4e88bdbae None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.601308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb9f47-faae-4354-85da-bfd4e88bdbae None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.603019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 182/360] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:38.603896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb9f47-faae-4354-85da-bfd4e88bdbae None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.609835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.610700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.610979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.611254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.613559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.615135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.617170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 179/361] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.617836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.729188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb9f47-faae-4354-85da-bfd4e88bdbae None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.732727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a129c-01e1-4c36-af8c-fbecda65bdc1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.752319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.772647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff0838c-5e68-4f43-976e-ab938648c21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.773534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0838c-5e68-4f43-976e-ab938648c21b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.773814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0838c-5e68-4f43-976e-ab938648c21b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.774088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff0838c-5e68-4f43-976e-ab938648c21b None admin] PATH_INFO: `/v3/users/7523e7cebce348349a764ae00ae50894` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.774459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.775121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.775345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0838c-5e68-4f43-976e-ab938648c21b None admin] RBAC: Authorizing `identity:get_user(user_id=7523e7cebce348349a764ae00ae50894)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.775437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.775670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.776907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=3418ad75d020471a91b2b3cffb565408) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:38.777023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff0838c-5e68-4f43-976e-ab938648c21b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.777107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.778346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.779512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 183/362] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/users/7523e7cebce348349a764ae00ae50894 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.780816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d42332-c4a8-4852-a6b5-c740731182a2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.781104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d42332-c4a8-4852-a6b5-c740731182a2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.781563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d42332-c4a8-4852-a6b5-c740731182a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.786101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:38.788275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 180/363] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.789148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.808837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f2d42332-c4a8-4852-a6b5-c740731182a2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:38.811225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f2d42332-c4a8-4852-a6b5-c740731182a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:38.924942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 184/364] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:38.926564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee5813-45bb-48fc-a862-3b4d5e47994c None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.926994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee5813-45bb-48fc-a862-3b4d5e47994c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.927410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee5813-45bb-48fc-a862-3b4d5e47994c None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.929406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 185/365] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:38.930337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee5813-45bb-48fc-a862-3b4d5e47994c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:38.943126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107de6a-2b65-4ef5-bd7c-4c4eb35fd1f5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:38.982805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20823fb-a466-406e-a66b-1d8f90370280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:38.983525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20823fb-a466-406e-a66b-1d8f90370280 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.983849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20823fb-a466-406e-a66b-1d8f90370280 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.984174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20823fb-a466-406e-a66b-1d8f90370280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:38.985020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20823fb-a466-406e-a66b-1d8f90370280 None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=47be6257c8c8422a87841e5a37f65973) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:38.985381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20823fb-a466-406e-a66b-1d8f90370280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:38.986887 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20823fb-a466-406e-a66b-1d8f90370280 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:38.993360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 181/366] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:38.994682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea776fd0-9e08-47e6-96ab-7c89db11fe59 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:38.995012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea776fd0-9e08-47e6-96ab-7c89db11fe59 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:38.995297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea776fd0-9e08-47e6-96ab-7c89db11fe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:39.014013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ea776fd0-9e08-47e6-96ab-7c89db11fe59 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:39.014919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ea776fd0-9e08-47e6-96ab-7c89db11fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:39.050147 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee5813-45bb-48fc-a862-3b4d5e47994c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:39.092523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:39.093455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:39.094073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:39.094988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:39.096032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] RBAC: Adding query filter params (role.id=ed5809ab31cb49e9b7122b1605a2e052, user.id=7523e7cebce348349a764ae00ae50894, scope.system=all) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:39.096495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:39.098502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:39.108726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:39.113572 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 186/367] 10.222.0.50 () {56 vars in 1308 bytes} [Thu Jun 30 15:37:38 2022] GET /identity/v3/role_assignments?user.id=7523e7cebce348349a764ae00ae50894&scope.system=all&role.id=ed5809ab31cb49e9b7122b1605a2e052 => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:39.120872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:39.137250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:39.147807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 182/368] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:39.181128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea776fd0-9e08-47e6-96ab-7c89db11fe59 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:39.274503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2989e-0f8a-4113-999f-1bf2ad3b65c2 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:39.323125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:39.323956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:39.324458 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:39.324769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:39.337419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:39.346725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:39.347651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:39.348480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 187/369] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:39.356415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:39.397593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:39.470974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141a78-b7ba-4b08-8a62-8f14474e8323 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:39.483972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea776fd0-9e08-47e6-96ab-7c89db11fe59 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:39.508718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:39.509401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:39.509682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:39.509981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:39.511028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:39.511339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:39.512949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:39.522614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 188/370] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:39.529386 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:39.540736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:39.541542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:39.541819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:39.542087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:39.561214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:39.569086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:39.570010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:39.570795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 183/371] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:39.576433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:39.640170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb2229-b206-4d1f-b5f3-ec877f9a350f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:39.652362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:39.667044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:39.692364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:39.694303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:39.694585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:39.694858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:39.750154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32264de8-1384-47d0-bb96-f2d64add8fac None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:39.793619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:39.795019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:39.810657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:39.811338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:39.811637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:39.811938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:39.812729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:39.813041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:39.814429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:39.834993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 184/372] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:39.838310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:39.955000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:39.964490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 189/373] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:39.967417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:40.016349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78aebb-1050-44fb-b99c-2357824c3fed None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:40.055617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:40.086152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:40.086816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.087099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.087374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.160046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7c95f-dafa-4759-b479-2835e53fff97 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:40.224304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:40.225775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:40.230758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:40.231649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.231927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.232226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.234216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:40.235690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:40.243045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 190/374] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:39 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:40.248571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:40.412444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:40.430604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 185/375] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:40.442032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:40.469735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:40.476230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f1cc26-4c08-4e46-b11e-5c35c9ef7484 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:40.526532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e1455-f264-4f5b-97db-5164ca3c9f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:40.527412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e1455-f264-4f5b-97db-5164ca3c9f3f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.527747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e1455-f264-4f5b-97db-5164ca3c9f3f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.530009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e1455-f264-4f5b-97db-5164ca3c9f3f None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.531407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e1455-f264-4f5b-97db-5164ca3c9f3f None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:40.532937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e1455-f264-4f5b-97db-5164ca3c9f3f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:40.535091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 191/376] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:40 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:40.536405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e092f-5bd2-4e58-b488-6d0c5f09dcbf None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.536694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e092f-5bd2-4e58-b488-6d0c5f09dcbf None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.536969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e092f-5bd2-4e58-b488-6d0c5f09dcbf None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.538217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 192/377] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:40.544636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e092f-5bd2-4e58-b488-6d0c5f09dcbf None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:40.569864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9d9121-e575-488b-afa6-e15942803230 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:40.614450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e38cf65-70dc-4609-892c-c9d45a38ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:40.615320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38cf65-70dc-4609-892c-c9d45a38ef7f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38cf65-70dc-4609-892c-c9d45a38ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.615872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e38cf65-70dc-4609-892c-c9d45a38ef7f None admin] PATH_INFO: `/v3/projects/052c2602e738443cb84400121f999c4f` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.617754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38cf65-70dc-4609-892c-c9d45a38ef7f None admin] RBAC: Authorizing `identity:get_project(project_id=052c2602e738443cb84400121f999c4f)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:40.619201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e38cf65-70dc-4609-892c-c9d45a38ef7f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:40.625504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 186/378] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:40 2022] GET /identity/v3/projects/052c2602e738443cb84400121f999c4f => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:40.626866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253216-177f-4fb4-9238-cfcb6e1fdf8f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.627147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253216-177f-4fb4-9238-cfcb6e1fdf8f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.627418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32253216-177f-4fb4-9238-cfcb6e1fdf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.649474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-32253216-177f-4fb4-9238-cfcb6e1fdf8f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:40.650667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-32253216-177f-4fb4-9238-cfcb6e1fdf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:40.664230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e092f-5bd2-4e58-b488-6d0c5f09dcbf None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:40.702952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:40.703622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.703908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.704318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.705165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=47be6257c8c8422a87841e5a37f65973) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:40.705479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:40.707051 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:40.714823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 193/379] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:40 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:40.715768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:40.720353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:40.732033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 187/380] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:40.762300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253216-177f-4fb4-9238-cfcb6e1fdf8f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:40.782034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:40.834998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115f8c5-c3fe-49ac-aa80-cbddfa08e77f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:40.871674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:40.872598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:40.872881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:40.873158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] PATH_INFO: `/v3/groups/dfc503d6b78d4a2d8725b69108aab086` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:40.878465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] RBAC: Authorizing `identity:get_group(group_id=dfc503d6b78d4a2d8725b69108aab086)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:40.879951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:40.882022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 194/381] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:40 2022] GET /identity/v3/groups/dfc503d6b78d4a2d8725b69108aab086 => generated 229 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:40.894914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:40.974049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32253216-177f-4fb4-9238-cfcb6e1fdf8f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:40.988406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:41.019959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f284005-5a51-49f3-a75d-3c87251c9e23 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:41.060661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:41.061999 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.062449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.062998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.064115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] RBAC: Adding query filter params (group.id=dfc503d6b78d4a2d8725b69108aab086, role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=052c2602e738443cb84400121f999c4f) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:41.064576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:41.066179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:41.073838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 195/382] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:40 2022] GET /identity/v3/role_assignments?group.id=dfc503d6b78d4a2d8725b69108aab086&scope.project.id=052c2602e738443cb84400121f999c4f&role.id=5c802c0e130549e2ba423dbef846a513 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:41.097376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:41.098211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.098511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.098793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.123959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:41.131903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:41.133039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:41.133849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:41.140578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 188/383] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:41.146670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:41.289394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55977b-cea7-4905-8daf-bf37a05fcd4f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:41.320122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:41.334232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:41.334931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.335216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.335507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.336315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:41.336624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:41.338007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:41.347804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 196/384] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:41.354073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:41.454796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:41.480800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f0798-891f-4e17-9665-4addd12d6893 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:41.515948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888a906-697e-407e-ace5-688f2e62d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:41.516848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a906-697e-407e-ace5-688f2e62d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.517133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a906-697e-407e-ace5-688f2e62d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.517414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888a906-697e-407e-ace5-688f2e62d7b1 None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.518654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888a906-697e-407e-ace5-688f2e62d7b1 None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:41.520089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888a906-697e-407e-ace5-688f2e62d7b1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:41.522041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 189/385] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:41 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:41.527666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:41.546458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185ec9-c3dc-4719-9bdf-076388d23e79 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.546764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185ec9-c3dc-4719-9bdf-076388d23e79 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.547009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5185ec9-c3dc-4719-9bdf-076388d23e79 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.548463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 190/386] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:41.553995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3602e-bb36-4fa5-acbb-7e90ff69caae None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.554473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3602e-bb36-4fa5-acbb-7e90ff69caae None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.554885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3602e-bb36-4fa5-acbb-7e90ff69caae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.579606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a0c3602e-bb36-4fa5-acbb-7e90ff69caae None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:41.582594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a0c3602e-bb36-4fa5-acbb-7e90ff69caae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:41.692478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ccc1f-0a0e-453d-8cf5-ed643734b311 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:41.699522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 191/387] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:41.704010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a41b3-5e40-40c6-983d-7e254d4f6160 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.704010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a41b3-5e40-40c6-983d-7e254d4f6160 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.704010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a41b3-5e40-40c6-983d-7e254d4f6160 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.704456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 192/388] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:41.709761 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314f68-32ca-4e1d-9b97-a383e6091709 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.710049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314f68-32ca-4e1d-9b97-a383e6091709 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.710325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314f68-32ca-4e1d-9b97-a383e6091709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.730209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ee314f68-32ca-4e1d-9b97-a383e6091709 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:41.731464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ee314f68-32ca-4e1d-9b97-a383e6091709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:41.754199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:41.755083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.755364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.756213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.756887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:41.758906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:41.764058 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 197/389] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:41 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:41.765168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:41.827281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:41.830952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 193/390] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:41.830952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314f68-32ca-4e1d-9b97-a383e6091709 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:41.851817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:41.894476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31163a18-4f9a-444d-8412-2915e0396eae None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:41.933643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:41.934529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:41.934811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:41.935084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:41.950102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:41.960129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:41.961067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:41.961879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:41.964728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 198/391] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:41.965784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:41.966822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314f68-32ca-4e1d-9b97-a383e6091709 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.008811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.009889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.012009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.012009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408/groups/d7f2c717931746acb339388718f0903b/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.012773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] RBAC: Authorizing `identity:create_grant(project_id=3418ad75d020471a91b2b3cffb565408, group_id=d7f2c717931746acb339388718f0903b, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.014451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.036037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 194/392] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Jun 30 15:37:41 2022] PUT /identity/v3/projects/3418ad75d020471a91b2b3cffb565408/groups/d7f2c717931746acb339388718f0903b/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:42.036037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:42.115686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a628b34-267f-4803-9acd-8e41638c849a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.139117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:42.162055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.163114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.163458 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.163838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.176089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:42.192094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:42.210815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:42.211790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:42.212632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:42.215518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 199/393] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:42.222703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:42.257128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf5acd-2d45-464c-8ec4-0a38ac3a6daa None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.301796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.303063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.303357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.303632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.304440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:42.304920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.306570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.317370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 195/394] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:42.323573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:42.438725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbb678-cc08-4543-aff7-fc858f277dd3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.450091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1581d9b-83b4-4331-8777-c2d546845c69 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.465282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:42.479441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.480928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.481359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.481775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.483161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.485295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.487531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 196/395] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:42 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:42.493574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:42.545274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.546150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.546534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.547089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.547759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] RBAC: Adding query filter params (name=member) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:42.548197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.549698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.566549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 200/396] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:42.586188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:42.615509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed77c430-ca30-4fb7-b5c9-a0ca71eeb944 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.654031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.655404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.655946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.656385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.658055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.659903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.662270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 197/397] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:42 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:42.667859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:42.672333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:42.736077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f45906-ce00-49c4-ad6c-02d904f5de94 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.738394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:42.773136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.774251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.774631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.775000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.776345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.778143 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.784014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 201/398] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:42 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:42.788046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e323a15-6aa1-48b6-a45c-d4d0ef86b311 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.793867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:42.833248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.834457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.834907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.835354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408/groups/d7f2c717931746acb339388718f0903b/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.841835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3418ad75d020471a91b2b3cffb565408, group_id=d7f2c717931746acb339388718f0903b, role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.843491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.868365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 198/399] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Jun 30 15:37:42 2022] PUT /identity/v3/projects/3418ad75d020471a91b2b3cffb565408/groups/d7f2c717931746acb339388718f0903b/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:37:42.917540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2d643-0342-4e0e-9cad-a1b3d9f870b7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:42.925781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:42.957083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e372b0-0aa3-4cf8-bd7c-20966248ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:42.958004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e372b0-0aa3-4cf8-bd7c-20966248ac64 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:42.958270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e372b0-0aa3-4cf8-bd7c-20966248ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:42.958544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e372b0-0aa3-4cf8-bd7c-20966248ac64 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:42.959841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e372b0-0aa3-4cf8-bd7c-20966248ac64 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:42.961338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e372b0-0aa3-4cf8-bd7c-20966248ac64 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:42.963522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 202/400] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:42 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:42.969151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:43.098495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9c00a-5dab-40c5-90bf-9196e6a63fd3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:43.120139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:43.138922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455c97f-e97e-46f6-9c81-42fbfa383ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:43.140312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c97f-e97e-46f6-9c81-42fbfa383ad8 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:43.140802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c97f-e97e-46f6-9c81-42fbfa383ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:43.141389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455c97f-e97e-46f6-9c81-42fbfa383ad8 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973/groups/d7f2c717931746acb339388718f0903b/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:43.143678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455c97f-e97e-46f6-9c81-42fbfa383ad8 None admin] RBAC: Authorizing `identity:create_grant(project_id=47be6257c8c8422a87841e5a37f65973, group_id=d7f2c717931746acb339388718f0903b, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:43.145414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455c97f-e97e-46f6-9c81-42fbfa383ad8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:43.166655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 199/401] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Jun 30 15:37:42 2022] PUT /identity/v3/projects/47be6257c8c8422a87841e5a37f65973/groups/d7f2c717931746acb339388718f0903b/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:43.627337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e40d79-3ab0-41e7-9f03-728c06975094 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:43.627908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e40d79-3ab0-41e7-9f03-728c06975094 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:43.627908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e40d79-3ab0-41e7-9f03-728c06975094 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:43.629550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 203/402] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:43.634624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d4f2d-36cf-48f8-b80e-0da861862782 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:43.635403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d4f2d-36cf-48f8-b80e-0da861862782 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:43.635484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d4f2d-36cf-48f8-b80e-0da861862782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:43.643865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18039308-419b-4ee1-bc2f-c7ebc78339b3 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:43.645139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18039308-419b-4ee1-bc2f-c7ebc78339b3 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:43.645425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18039308-419b-4ee1-bc2f-c7ebc78339b3 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:43.647045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 204/403] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:43.656632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d3053-9c2c-41a9-8e35-78354ae3871a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:43.656908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d3053-9c2c-41a9-8e35-78354ae3871a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:43.657190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d3053-9c2c-41a9-8e35-78354ae3871a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:43.658802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9b5d4f2d-36cf-48f8-b80e-0da861862782 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:43.660076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9b5d4f2d-36cf-48f8-b80e-0da861862782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:43.692314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c06d3053-9c2c-41a9-8e35-78354ae3871a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:43.693502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c06d3053-9c2c-41a9-8e35-78354ae3871a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:43.790728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 200/404] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:43.802660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 205/405] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:43.815029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d4f2d-36cf-48f8-b80e-0da861862782 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:43.838662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d3053-9c2c-41a9-8e35-78354ae3871a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:43.968519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d4f2d-36cf-48f8-b80e-0da861862782 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:43.990493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:44.024182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.025003 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.025281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.025554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.029874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d3053-9c2c-41a9-8e35-78354ae3871a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.032465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:44.046265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:44.056545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:44.058783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:44.064800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 201/406] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:44.066662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.091506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.092357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.092639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.092912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.120618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:44.128485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:44.130683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:44.133007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 206/407] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:44.144025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.211197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a112-7ccc-4d69-9989-a2687ae297b8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.249981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.251031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.251524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.252080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.253042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:44.253553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:44.255246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:44.265316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 202/408] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:44.277269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.311152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503cb07-9c5f-4607-b715-97b5e30aa912 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.346819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.347498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.347781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.348087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.348903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:44.349215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:44.350652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:44.357133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:44.360350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 207/409] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:44.368904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.423747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:44.475978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d4cf0-abd4-4dad-a02c-8a2268782451 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.535719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.536529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.536808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.537947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.538353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:44.539831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:44.548664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 203/410] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:44 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:44.550152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.573123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85aaa75-2f5a-430b-adee-64dc9b243990 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.629995 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e101f-305a-4769-a883-cb0b4c4ff28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.631198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e101f-305a-4769-a883-cb0b4c4ff28f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.631611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e101f-305a-4769-a883-cb0b4c4ff28f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.632036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e101f-305a-4769-a883-cb0b4c4ff28f None admin] PATH_INFO: `/v3/groups/dfc503d6b78d4a2d8725b69108aab086` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.633238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e101f-305a-4769-a883-cb0b4c4ff28f None admin] RBAC: Authorizing `identity:get_group(group_id=dfc503d6b78d4a2d8725b69108aab086)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:44.635188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e101f-305a-4769-a883-cb0b4c4ff28f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:44.637508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 208/411] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:44 2022] GET /identity/v3/groups/dfc503d6b78d4a2d8725b69108aab086 => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:44.638729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33bcc5-3240-4575-9fe5-665c0ad3942f None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.639012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33bcc5-3240-4575-9fe5-665c0ad3942f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.639287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33bcc5-3240-4575-9fe5-665c0ad3942f None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.640790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 209/412] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:44.646127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8a9b9-0868-4204-b697-bb32be37780a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.646415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8a9b9-0868-4204-b697-bb32be37780a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.646694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8a9b9-0868-4204-b697-bb32be37780a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.674930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-43e8a9b9-0868-4204-b697-bb32be37780a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:44.676362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-43e8a9b9-0868-4204-b697-bb32be37780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:44.700039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:44.703409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb303b36-e30c-4ad4-a69c-54eb96b8b60f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.746410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.747285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.747562 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.747832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.749071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:44.750533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:44.753068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 204/413] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:44 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:44.753848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.773678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 210/414] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:44.774948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8a9b9-0868-4204-b697-bb32be37780a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.816834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:44.907439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62953d1d-0bfd-492a-9320-d114b733cbde None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.915103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8a9b9-0868-4204-b697-bb32be37780a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:44.957194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.958059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.958340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.958615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] PATH_INFO: `/v3/projects/052c2602e738443cb84400121f999c4f` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.959372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:44.959809 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] RBAC: Authorizing `identity:get_project(project_id=052c2602e738443cb84400121f999c4f)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:44.960739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:44.961226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:44.961326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:44.961859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973/groups/d7f2c717931746acb339388718f0903b/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:44.963307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 205/415] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:44 2022] GET /identity/v3/projects/052c2602e738443cb84400121f999c4f => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:44.965184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:44.966090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] RBAC: Authorizing `identity:create_grant(project_id=47be6257c8c8422a87841e5a37f65973, group_id=d7f2c717931746acb339388718f0903b, role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:44.971649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:44.996033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 211/416] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Jun 30 15:37:44 2022] PUT /identity/v3/projects/47be6257c8c8422a87841e5a37f65973/groups/d7f2c717931746acb339388718f0903b/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:44.996033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:45.041153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:45.161257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:45.165389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafeb160-b5b7-41d0-8b08-ba7d1cfcdf23 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:45.195672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce280e5-53f8-458f-9df7-40a5a0e17542 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:45.235062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:45.235860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.236168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.236443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.250611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:45.252226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.252358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:45.252881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.253352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] PATH_INFO: `/v3/projects/052c2602e738443cb84400121f999c4f/groups/dfc503d6b78d4a2d8725b69108aab086/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.255715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=052c2602e738443cb84400121f999c4f, group_id=dfc503d6b78d4a2d8725b69108aab086, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:45.257394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:45.260633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:45.261520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:45.262862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 206/417] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:45.267714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:45.282106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 212/418] 10.222.0.50 () {58 vars in 1442 bytes} [Thu Jun 30 15:37:44 2022] PUT /identity/v3/projects/052c2602e738443cb84400121f999c4f/groups/dfc503d6b78d4a2d8725b69108aab086/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:37:45.381220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:45.389244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6a70a-fdf0-41be-9a8e-01b2c298a8ce None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:45.440118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581e0bb-3904-461d-bab3-251a8d8320cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:45.440704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e0bb-3904-461d-bab3-251a8d8320cf None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.440990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e0bb-3904-461d-bab3-251a8d8320cf None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.441269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581e0bb-3904-461d-bab3-251a8d8320cf None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.442063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581e0bb-3904-461d-bab3-251a8d8320cf None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:45.442372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581e0bb-3904-461d-bab3-251a8d8320cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:45.443808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581e0bb-3904-461d-bab3-251a8d8320cf None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:45.454251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 207/419] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:45.460273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:45.570263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc14a17-643d-4d7f-b18e-440471bee0d4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:45.578354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcaf6-2fd7-4553-8d09-213abc68882f None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.578971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcaf6-2fd7-4553-8d09-213abc68882f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.579457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdcaf6-2fd7-4553-8d09-213abc68882f None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.581380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 208/420] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:45.589114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71edbd-1b5c-4837-aa16-b37ac1a4f1f7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.589687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71edbd-1b5c-4837-aa16-b37ac1a4f1f7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.590275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71edbd-1b5c-4837-aa16-b37ac1a4f1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.611152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9e71edbd-1b5c-4837-aa16-b37ac1a4f1f7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:45.613553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9e71edbd-1b5c-4837-aa16-b37ac1a4f1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:45.632521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0146ecc-3f99-4bda-9691-b4a5529019dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:45.633204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0146ecc-3f99-4bda-9691-b4a5529019dc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.633463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0146ecc-3f99-4bda-9691-b4a5529019dc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.633737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0146ecc-3f99-4bda-9691-b4a5529019dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.664712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:45.732277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 209/421] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:45.772303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71edbd-1b5c-4837-aa16-b37ac1a4f1f7 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:45.784335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0146ecc-3f99-4bda-9691-b4a5529019dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:45.786278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0146ecc-3f99-4bda-9691-b4a5529019dc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:45.919957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:45.932545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71edbd-1b5c-4837-aa16-b37ac1a4f1f7 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:45.953389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 213/422] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:45.958093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d259d8-1aa9-4f98-9660-a4473b4d2956 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.958093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d259d8-1aa9-4f98-9660-a4473b4d2956 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.958093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d259d8-1aa9-4f98-9660-a4473b4d2956 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.960366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 214/423] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:45.961231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d259d8-1aa9-4f98-9660-a4473b4d2956 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:45.981386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb27e37-2deb-4cd7-9445-576852dd8b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:45.982665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27e37-2deb-4cd7-9445-576852dd8b65 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:45.983158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27e37-2deb-4cd7-9445-576852dd8b65 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:45.983643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27e37-2deb-4cd7-9445-576852dd8b65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:45.997458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb27e37-2deb-4cd7-9445-576852dd8b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5fb27e37-2deb-4cd7-9445-576852dd8b65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:46.006014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:46.006938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:46.007742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:46.012395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 210/424] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:46.013724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ccfcd-a423-433f-aa60-e6088fb00045 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.014011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ccfcd-a423-433f-aa60-e6088fb00045 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.014287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ccfcd-a423-433f-aa60-e6088fb00045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.023313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:46.033909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c94ccfcd-a423-433f-aa60-e6088fb00045 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:46.035105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c94ccfcd-a423-433f-aa60-e6088fb00045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:46.096520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d259d8-1aa9-4f98-9660-a4473b4d2956 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.120329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 211/425] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:46.121345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ccfcd-a423-433f-aa60-e6088fb00045 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.141362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.143065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.143909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.144431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.146010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:46.147699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:46.150429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 215/426] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:45 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:46.156652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.224024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:46.247363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ccfcd-a423-433f-aa60-e6088fb00045 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.289279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.289958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.290241 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.290574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.291290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:46.291594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:46.292992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e09dda-e5f7-42fe-99cf-7c7cfeab702c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.293113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:46.308124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 212/427] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:46.309204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:46.310059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.336415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.338124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.338813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.339249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.352967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:46.363802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:46.364962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 15:37:46.365946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:46.368828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 216/428] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:46.369757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.430384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22369c6-0921-4194-b04b-1326d7f74b61 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.468431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.468865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.469146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.469416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.470628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:46.472102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:46.478705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 213/429] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:46.478705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.490750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:46.513903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ebc0c-8859-4083-8ce5-5b60e60dba6d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.557623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.559190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.559766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.560318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.596556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a2a94-69a1-4df3-8cde-15e2ea63329a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.616781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:46.639821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.640924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.641290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.641636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.642720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] RBAC: Adding query filter params (name=member) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:46.643096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:46.645427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:46.650486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:46.652635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:46.656468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 214/430] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:46.662057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.752358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:46.761415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 217/431] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:46.761415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.801215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49e1b8-8c06-4f58-a9f9-add08373c09b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.838474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:46.850702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.851969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.852933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.853680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:46.854983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=3418ad75d020471a91b2b3cffb565408) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:46.855978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:46.857690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:46.866813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 215/432] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:46.867836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:46.882758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2572d-60c3-46f7-820f-99fa9b376c9d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:46.973496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:46.974530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:46.976876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:46.977483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.066670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:47.069280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:47.089560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef00f55-1241-4d0e-85b9-bf0d24fd7ec8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.126962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.127830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.128140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.128417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] PATH_INFO: `/v3/projects/3418ad75d020471a91b2b3cffb565408` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.129653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] RBAC: Authorizing `identity:get_project(project_id=3418ad75d020471a91b2b3cffb565408)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:47.131225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:47.133186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 216/433] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/projects/3418ad75d020471a91b2b3cffb565408 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:47.144875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.147849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:47.163726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:47.200022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 218/434] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:47.208116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.275826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897e437-3684-4535-8c1c-2865fd8bfcff None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.314961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.315882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.316210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.316507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.317800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:47.319221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:47.321234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 217/435] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:47.327071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.336394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e009e4-52ad-4fab-bddb-0c3683ec1693 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.384242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf1c49-4763-42f0-af0f-bcf6f996fc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.385428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf1c49-4763-42f0-af0f-bcf6f996fc4d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.385711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf1c49-4763-42f0-af0f-bcf6f996fc4d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.385987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf1c49-4763-42f0-af0f-bcf6f996fc4d None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.387299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf1c49-4763-42f0-af0f-bcf6f996fc4d None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:47.388790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf1c49-4763-42f0-af0f-bcf6f996fc4d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:47.390806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 219/436] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:47.392048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5e32a-d6f5-431c-bcdd-ff458dd2dc15 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.392336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5e32a-d6f5-431c-bcdd-ff458dd2dc15 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.392615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5e32a-d6f5-431c-bcdd-ff458dd2dc15 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.394079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 220/437] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:47.398781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5e32a-d6f5-431c-bcdd-ff458dd2dc15 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.412850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:47.446243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de7295-1822-4013-bc41-3fe6feaa1123 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.454891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:47.494085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121afc64-b093-49b0-9c94-3528c2c1f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.494769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121afc64-b093-49b0-9c94-3528c2c1f89a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.495072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121afc64-b093-49b0-9c94-3528c2c1f89a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.495342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121afc64-b093-49b0-9c94-3528c2c1f89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.496242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121afc64-b093-49b0-9c94-3528c2c1f89a None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=3418ad75d020471a91b2b3cffb565408) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:47.496557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121afc64-b093-49b0-9c94-3528c2c1f89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:47.500234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121afc64-b093-49b0-9c94-3528c2c1f89a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:47.507207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 218/438] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=3418ad75d020471a91b2b3cffb565408&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:47.510980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945a3e-c7ff-4d4e-a05c-1b730b32e8d5 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.510980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945a3e-c7ff-4d4e-a05c-1b730b32e8d5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.511249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945a3e-c7ff-4d4e-a05c-1b730b32e8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.531952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5e32a-d6f5-431c-bcdd-ff458dd2dc15 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.534647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-46945a3e-c7ff-4d4e-a05c-1b730b32e8d5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:47.536102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-46945a3e-c7ff-4d4e-a05c-1b730b32e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:47.577201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d622b2b-892a-41f7-9c1e-834c6dd189dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.579359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622b2b-892a-41f7-9c1e-834c6dd189dc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.579639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622b2b-892a-41f7-9c1e-834c6dd189dc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.579915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d622b2b-892a-41f7-9c1e-834c6dd189dc None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.582235 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d622b2b-892a-41f7-9c1e-834c6dd189dc None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:47.583773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d622b2b-892a-41f7-9c1e-834c6dd189dc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:47.585756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 221/439] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:47.586939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb721f6-49c1-40cf-8fbb-ccb2479a435b None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.587221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb721f6-49c1-40cf-8fbb-ccb2479a435b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.588284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb721f6-49c1-40cf-8fbb-ccb2479a435b None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.593436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 222/440] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:47.595350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb721f6-49c1-40cf-8fbb-ccb2479a435b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.632444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 219/441] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:47.633801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b406e9-2e03-4042-b79c-3e40f4ae1f01 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.634089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b406e9-2e03-4042-b79c-3e40f4ae1f01 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.634364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b406e9-2e03-4042-b79c-3e40f4ae1f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.660034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c1b406e9-2e03-4042-b79c-3e40f4ae1f01 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:47.661254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c1b406e9-2e03-4042-b79c-3e40f4ae1f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:47.725868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb721f6-49c1-40cf-8fbb-ccb2479a435b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.726425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:47.736881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:47.740356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 220/442] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:47.743344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b406e9-2e03-4042-b79c-3e40f4ae1f01 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.776940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.778491 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.778788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.779064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.779902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=fd1cdca617c24c089cbe498be4cd05f8, scope.project.id=47be6257c8c8422a87841e5a37f65973) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:47.780278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:47.781746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:47.789815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 223/443] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=fd1cdca617c24c089cbe498be4cd05f8 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:47.797196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.865976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b406e9-2e03-4042-b79c-3e40f4ae1f01 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.919673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.921073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.921357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.921738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:47.937623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:47.946380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:47.947383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:47.948695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:47.949804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd199ca-dae2-409a-bd68-a0d63d86f2e1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:47.949879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 221/444] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:47.954557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:47.990289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:47.991102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:47.991384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:47.991662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.003935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:48.014617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:48.015593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:37:48.016540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:48.020078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 224/445] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:48.022488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.031000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.059255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.081786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02345ba1-867a-4cd0-8dcb-28070fd54721 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.130955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.131565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.131862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.132599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.133367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:48.133695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.135414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.144120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 222/446] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:48.147546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957302f-a086-451e-8e75-82fd070d1c88 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.150752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.187500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.188279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.188557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.188826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.189596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:48.189899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.191286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.199833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 225/447] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:48.205621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.250508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080d2fc-67ea-4b75-a048-9f427f18086b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.269416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.288266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.288949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.289227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.289484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.301837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.308976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665f380-4b31-49ca-8a3a-541382159efe None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.340962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.341610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.341893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.342171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.355150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.356734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.408606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.409979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.456789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 223/448] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:48.459529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.476625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.497848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.506181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 226/449] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 15:37:48.511561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.557039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb2c69-ac78-4641-bb58-e867a46b9cf1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.595422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.596343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.596618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.596888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] PATH_INFO: `/v3/projects/47be6257c8c8422a87841e5a37f65973` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.598089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=47be6257c8c8422a87841e5a37f65973)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.599574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.601333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 224/450] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/projects/47be6257c8c8422a87841e5a37f65973 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:48.606856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.616117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3dc0e-47b1-4b81-87f2-fa2cb938bb3e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.650993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.651863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.652178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.652572 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] PATH_INFO: `/v3/projects/052c2602e738443cb84400121f999c4f` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.653833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] RBAC: Authorizing `identity:get_project(project_id=052c2602e738443cb84400121f999c4f)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.655287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.656986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 227/451] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/projects/052c2602e738443cb84400121f999c4f => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:48.662641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.687719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.704190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2af08e-3c72-4bb9-9949-720f94070fb0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.722343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.738000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.738895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.739174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.739448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] PATH_INFO: `/v3/groups/d7f2c717931746acb339388718f0903b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.740625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] RBAC: Authorizing `identity:get_group(group_id=d7f2c717931746acb339388718f0903b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.742075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.743730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 225/452] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/groups/d7f2c717931746acb339388718f0903b => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:48.749633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.762970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd41fa-f959-4301-9b52-adeee5227214 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.796301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.797218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.797497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.797775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] PATH_INFO: `/v3/groups/dfc503d6b78d4a2d8725b69108aab086` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.798984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] RBAC: Authorizing `identity:get_group(group_id=dfc503d6b78d4a2d8725b69108aab086)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.800556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.802261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 228/453] 10.222.0.50 () {56 vars in 1180 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/groups/dfc503d6b78d4a2d8725b69108aab086 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:48.808358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:48.851040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33814e-60ad-4b6a-84a5-88674e50c60b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.883770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e00379-b6c2-4fec-a3e6-bd201e25a959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.884485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e00379-b6c2-4fec-a3e6-bd201e25a959 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.885044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e00379-b6c2-4fec-a3e6-bd201e25a959 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.885319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e00379-b6c2-4fec-a3e6-bd201e25a959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.886153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e00379-b6c2-4fec-a3e6-bd201e25a959 None admin] RBAC: Adding query filter params (group.id=d7f2c717931746acb339388718f0903b, role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=47be6257c8c8422a87841e5a37f65973) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:48.886459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e00379-b6c2-4fec-a3e6-bd201e25a959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.887968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e00379-b6c2-4fec-a3e6-bd201e25a959 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.893927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 226/454] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/role_assignments?group.id=d7f2c717931746acb339388718f0903b&scope.project.id=47be6257c8c8422a87841e5a37f65973&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:48.909322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1febb5-fa3b-47ad-a0e8-bdedd6596146 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:48.942801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:48.949600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce259ff6-941a-4e1c-a129-8123c9ecc53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:48.950275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce259ff6-941a-4e1c-a129-8123c9ecc53d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:48.950554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce259ff6-941a-4e1c-a129-8123c9ecc53d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:48.950828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce259ff6-941a-4e1c-a129-8123c9ecc53d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:48.951648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce259ff6-941a-4e1c-a129-8123c9ecc53d None admin] RBAC: Adding query filter params (group.id=dfc503d6b78d4a2d8725b69108aab086, role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=052c2602e738443cb84400121f999c4f) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:48.951981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce259ff6-941a-4e1c-a129-8123c9ecc53d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:48.953478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce259ff6-941a-4e1c-a129-8123c9ecc53d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:48.960202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 229/455] 10.222.0.50 () {56 vars in 1376 bytes} [Thu Jun 30 15:37:48 2022] GET /identity/v3/role_assignments?group.id=dfc503d6b78d4a2d8725b69108aab086&scope.project.id=052c2602e738443cb84400121f999c4f&role.id=5c802c0e130549e2ba423dbef846a513 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:51.719677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94087afa-ddf4-4478-ae4c-b77642f3a200 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:51.720566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94087afa-ddf4-4478-ae4c-b77642f3a200 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:51.720983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94087afa-ddf4-4478-ae4c-b77642f3a200 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:51.722499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 227/456] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:51.727878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004e16-ab07-4787-8099-1b4148f0f2e2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:51.728636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004e16-ab07-4787-8099-1b4148f0f2e2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:51.729074 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4004e16-ab07-4787-8099-1b4148f0f2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:51.747082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a3bb4-319d-4a36-9095-640a79bad3d1 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:51.747700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a3bb4-319d-4a36-9095-640a79bad3d1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:51.748117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f4004e16-ab07-4787-8099-1b4148f0f2e2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:51.748647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a3bb4-319d-4a36-9095-640a79bad3d1 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:51.749324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f4004e16-ab07-4787-8099-1b4148f0f2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:51.750746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 228/457] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:37:51.756853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510a2e-2143-434e-aae1-033be024a3cb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:51.757407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510a2e-2143-434e-aae1-033be024a3cb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:51.758010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510a2e-2143-434e-aae1-033be024a3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:51.789137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7e510a2e-2143-434e-aae1-033be024a3cb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:51.790623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7e510a2e-2143-434e-aae1-033be024a3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:51.812899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:51.848515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 230/458] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:51.896157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004e16-ab07-4787-8099-1b4148f0f2e2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:51.904240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 229/459] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:51.906068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb4ca-4f58-45cf-aacf-b6ecaa10d6e4 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:51.906360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb4ca-4f58-45cf-aacf-b6ecaa10d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:51.906641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fb4ca-4f58-45cf-aacf-b6ecaa10d6e4 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:51.911410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 230/460] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:51.913760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25574f9-a32c-4e6a-8be3-f030b0045ccc None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:51.913954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25574f9-a32c-4e6a-8be3-f030b0045ccc None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:51.914185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25574f9-a32c-4e6a-8be3-f030b0045ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:51.940817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c25574f9-a32c-4e6a-8be3-f030b0045ccc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:51.942065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c25574f9-a32c-4e6a-8be3-f030b0045ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:52.022045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4004e16-ab07-4787-8099-1b4148f0f2e2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.023165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 231/461] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:52.032134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25574f9-a32c-4e6a-8be3-f030b0045ccc None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.054003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.068539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.069634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.070126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.070567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.082886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.084753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:52.105690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:52.106838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:52.107773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:52.107773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:52.107773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.107773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:52.110312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 231/462] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:52.115781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.127597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.157688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25574f9-a32c-4e6a-8be3-f030b0045ccc None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.200540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.201573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.201863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.202100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.213437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.214948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.228798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbcf8f-0d5b-4846-ba91-ba9cfd15994c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:52.233821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:52.234669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:52.235509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:52.235509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:52.235509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.235509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:52.235509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 232/463] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:52.236326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.275470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbccd12-d070-46a0-b582-937159110f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.276256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbccd12-d070-46a0-b582-937159110f95 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.276535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbccd12-d070-46a0-b582-937159110f95 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.276806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbccd12-d070-46a0-b582-937159110f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.290644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbccd12-d070-46a0-b582-937159110f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.292156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbccd12-d070-46a0-b582-937159110f95 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-2fbccd12-d070-46a0-b582-937159110f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:52.309836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:52.310750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:52.311725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:52.311725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:52.311725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.311725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:52.311725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 232/464] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:52.312575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e50b6-fe5e-44b9-869b-d688c883624b None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.312575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e50b6-fe5e-44b9-869b-d688c883624b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.312826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e50b6-fe5e-44b9-869b-d688c883624b None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.319718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 233/465] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:52.319718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e50b6-fe5e-44b9-869b-d688c883624b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.345183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.350760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7cced-92f1-42a0-a22e-d24c5ad2ad40 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.380474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.388921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.389595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.389875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.390147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.390872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:52.391175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.392608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.405207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 233/466] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:52.405834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.429171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e50b6-fe5e-44b9-869b-d688c883624b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.465441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4a836-669f-4f22-a667-c2d2655e36f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.466113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4a836-669f-4f22-a667-c2d2655e36f5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.466392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4a836-669f-4f22-a667-c2d2655e36f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.466832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4a836-669f-4f22-a667-c2d2655e36f5 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.467480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4a836-669f-4f22-a667-c2d2655e36f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:52.467694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4a836-669f-4f22-a667-c2d2655e36f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.469103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4a836-669f-4f22-a667-c2d2655e36f5 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.480992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 234/467] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:52.487114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320f662-529c-402a-b139-8d9de88ea558 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.487543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320f662-529c-402a-b139-8d9de88ea558 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.487974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320f662-529c-402a-b139-8d9de88ea558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.504770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4320f662-529c-402a-b139-8d9de88ea558 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:52.505966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4320f662-529c-402a-b139-8d9de88ea558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:52.508477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ff279-b6f3-4df9-903c-35fcb161dec7 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.543214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.543805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.544113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.544388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.545109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:52.545419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.546810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.558261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 234/468] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:52.559108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.581985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 235/469] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:52.583201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320f662-529c-402a-b139-8d9de88ea558 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.610337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.627373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.686757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55d7e9-3807-4494-a2a4-0b1d31bcb617 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.701492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320f662-529c-402a-b139-8d9de88ea558 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.719909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.720608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.720890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.721167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.722284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.723762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.735175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.735849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.736158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.736542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.737264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.738697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.744189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 235/470] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:52 2022] POST /identity/v3/users => generated 246 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:52.745065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.759414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 236/471] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:52 2022] POST /identity/v3/users => generated 248 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:52.760489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.852949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.886945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb49e6e-2763-4858-88af-1e68027bd4a4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.899188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43a646-eab0-4c23-8ba6-1a6846868ecd None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:52.904163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:52.932541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622c59a-7a28-47ca-9cf6-020e3a290379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.933544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c59a-7a28-47ca-9cf6-020e3a290379 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.933861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c59a-7a28-47ca-9cf6-020e3a290379 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.934138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c59a-7a28-47ca-9cf6-020e3a290379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.946898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622c59a-7a28-47ca-9cf6-020e3a290379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.948392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622c59a-7a28-47ca-9cf6-020e3a290379 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.956250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea82ea-25cb-4740-92db-157079327963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:52.957109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea82ea-25cb-4740-92db-157079327963 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.957436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea82ea-25cb-4740-92db-157079327963 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.957709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea82ea-25cb-4740-92db-157079327963 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.958425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea82ea-25cb-4740-92db-157079327963 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:52.959869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea82ea-25cb-4740-92db-157079327963 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-1622c59a-7a28-47ca-9cf6-020e3a290379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:52.968964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:52.969886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:52.970742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:52.970742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:52.970742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:52.970742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:52.970950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 237/472] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:52.972267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4aa7c-ac7a-49fa-a6d7-cc670e21f6ba None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.972577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4aa7c-ac7a-49fa-a6d7-cc670e21f6ba None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.972864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4aa7c-ac7a-49fa-a6d7-cc670e21f6ba None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:52.976055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 238/473] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:52.978085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4aa7c-ac7a-49fa-a6d7-cc670e21f6ba None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:52.987786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 236/474] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:52 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:52.989410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a70be-ffef-4946-8fdc-64b11e4a97b6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:52.989840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a70be-ffef-4946-8fdc-64b11e4a97b6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:52.990361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a70be-ffef-4946-8fdc-64b11e4a97b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:53.015166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-855a70be-ffef-4946-8fdc-64b11e4a97b6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:53.016756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-855a70be-ffef-4946-8fdc-64b11e4a97b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:53.113862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 237/475] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:53.128668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4aa7c-ac7a-49fa-a6d7-cc670e21f6ba None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:53.144941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a70be-ffef-4946-8fdc-64b11e4a97b6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:53.158887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:53.169169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:53.170293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:53.170942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:53.171300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:53.172719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:53.173112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:53.174986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:53.190837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 239/476] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:37:53.201197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:53.290402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:53.304765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a70be-ffef-4946-8fdc-64b11e4a97b6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:53.334415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac8af6-737e-4799-88e9-e372ed971e3e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:53.344605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:53.345280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:53.345561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:53.345827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:53.346579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:53.347969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:53.358021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.sql.core [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 15:37:53.358021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 15:37:53.358021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [parameters: {'id': 'a6b84b3da09d4a2bbe3a298f12389b0f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 15:37:53.358021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91957) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application conn.query(q) Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result.read() Jun 30 15:37:53.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 15:37:53.379378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application next(self.gen) Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application yield resource Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application next(self.gen) Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise self.value Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application session.commit() Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 15:37:53.380349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.session.flush() Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application compat.raise_( Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 15:37:53.381320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 15:37:53.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application conn.query(q) Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result.read() Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 15:37:53.383032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [parameters: {'id': 'a6b84b3da09d4a2bbe3a298f12389b0f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:53.383824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 15:37:53.387117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:53.389182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 238/477] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Jun 30 15:37:53 2022] POST /identity/v3/roles => generated 144 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:53.389182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:53.391700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:53.392712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:53.393191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:53.393671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:53.394589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:53.396366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:53.425780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 240/478] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:53 2022] POST /identity/v3/users => generated 249 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:53.519850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fd57f-7b60-494c-846f-46acecd94d30 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:53.560388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5f4cd-ae08-4dc1-a81c-e0fa8dd2f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:53.561229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5f4cd-ae08-4dc1-a81c-e0fa8dd2f776 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:53.561511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5f4cd-ae08-4dc1-a81c-e0fa8dd2f776 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:53.561786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5f4cd-ae08-4dc1-a81c-e0fa8dd2f776 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:53.578945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5f4cd-ae08-4dc1-a81c-e0fa8dd2f776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-eef5f4cd-ae08-4dc1-a81c-e0fa8dd2f776 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:53.587445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:53.588446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:53.589235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 239/479] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:37:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:53.595318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:53.657480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:53.717986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e430a233-c1af-4535-ad79-96f0d056a9bb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:53.758265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da35702-8648-4f4f-a481-ffc0e006ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:53.759974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da35702-8648-4f4f-a481-ffc0e006ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:53.760448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da35702-8648-4f4f-a481-ffc0e006ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:53.760874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da35702-8648-4f4f-a481-ffc0e006ddc2 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:53.761817 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da35702-8648-4f4f-a481-ffc0e006ddc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:53.762273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da35702-8648-4f4f-a481-ffc0e006ddc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:53.763829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da35702-8648-4f4f-a481-ffc0e006ddc2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:53.772837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 241/480] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:55.222362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa172d-40f3-4f6c-9c31-ac3af89b3932 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.222889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa172d-40f3-4f6c-9c31-ac3af89b3932 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.222889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa172d-40f3-4f6c-9c31-ac3af89b3932 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.224568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 240/481] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:55.229904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e61a-93d0-4a8c-b4a0-13038e4e740a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.230335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e61a-93d0-4a8c-b4a0-13038e4e740a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.230535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2e61a-93d0-4a8c-b4a0-13038e4e740a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.252492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-abd2e61a-93d0-4a8c-b4a0-13038e4e740a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:55.253810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-abd2e61a-93d0-4a8c-b4a0-13038e4e740a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:55.254220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116998-0ba3-4395-89ee-5a95d6ae9f12 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.254708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116998-0ba3-4395-89ee-5a95d6ae9f12 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.255181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116998-0ba3-4395-89ee-5a95d6ae9f12 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.260365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 241/482] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:55.274371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54fdc-53a3-4c1b-b0b1-7b90f9337276 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.274663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54fdc-53a3-4c1b-b0b1-7b90f9337276 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.275052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54fdc-53a3-4c1b-b0b1-7b90f9337276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.317049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f1a54fdc-53a3-4c1b-b0b1-7b90f9337276 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:55.318509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f1a54fdc-53a3-4c1b-b0b1-7b90f9337276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:55.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 242/483] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:37:55.334420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:55.356897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e61a-93d0-4a8c-b4a0-13038e4e740a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:55.403304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:55.450636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 242/484] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:55.479115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2e61a-93d0-4a8c-b4a0-13038e4e740a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:55.507828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a54fdc-53a3-4c1b-b0b1-7b90f9337276 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:55.519762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:55.520611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.520894 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.521169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.532017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:55.539790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:55.540710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:55.541555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 243/485] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:37:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:37:55.546761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:55.683014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:55.692468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b73863-abb2-4faa-b8e5-dfdeaa5d88a8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:55.713566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a54fdc-53a3-4c1b-b0b1-7b90f9337276 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:55.735477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5ab18-84f2-4047-8f57-b41eb57f8701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:55.736135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5ab18-84f2-4047-8f57-b41eb57f8701 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.736414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5ab18-84f2-4047-8f57-b41eb57f8701 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.736685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5ab18-84f2-4047-8f57-b41eb57f8701 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.737436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5ab18-84f2-4047-8f57-b41eb57f8701 None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:55.737732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5ab18-84f2-4047-8f57-b41eb57f8701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:55.739114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5ab18-84f2-4047-8f57-b41eb57f8701 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:55.748189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 244/486] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:55.749931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff66a4-044e-4240-b98e-5821027c6a36 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.750273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff66a4-044e-4240-b98e-5821027c6a36 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.750548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff66a4-044e-4240-b98e-5821027c6a36 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.752591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3322ad5-6a29-4fed-8875-6469027106c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:55.753595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322ad5-6a29-4fed-8875-6469027106c8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.753994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322ad5-6a29-4fed-8875-6469027106c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.754366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3322ad5-6a29-4fed-8875-6469027106c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.756085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 245/487] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:55.756863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff66a4-044e-4240-b98e-5821027c6a36 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:55.767828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3322ad5-6a29-4fed-8875-6469027106c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-d3322ad5-6a29-4fed-8875-6469027106c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:55.778049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:55.779094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:55.779884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:55.780793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 243/488] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:37:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:55.782385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1123-9c44-4904-853e-cfdfedca1335 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.782679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1123-9c44-4904-853e-cfdfedca1335 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.782956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c1123-9c44-4904-853e-cfdfedca1335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.791736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:55.806534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-974c1123-9c44-4904-853e-cfdfedca1335 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:55.808211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-974c1123-9c44-4904-853e-cfdfedca1335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:55.871559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff66a4-044e-4240-b98e-5821027c6a36 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:55.898012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 244/489] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:55.899226 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1123-9c44-4904-853e-cfdfedca1335 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:55.922989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:55.924146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:55.924537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:55.924886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:55.945993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:55.947823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:55.966476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:55.967617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:55.968739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:55.968739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:55.968739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:55.968739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:55.969558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 246/490] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:55.970459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:55.996510 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.020808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c1123-9c44-4904-853e-cfdfedca1335 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.065922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.067176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.067608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.068228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.069012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:56.069330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.070739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.072242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.087022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 245/491] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:56.088274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.093335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945e9ed-11c2-4417-b884-7cf42b72d980 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.127955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.128837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.129151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.129426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.140630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:56.150166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:56.151086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:56.152624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 247/492] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:37:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:56.153318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.226426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040569b1-cac2-4db6-8397-57268941bfc3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.228377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.262145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.262825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.263000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.263272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.264035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:56.264334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.266118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.278023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389768d4-8d9e-4d76-ad96-3a2e15798799 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.285740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 246/493] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:56.286476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.319899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cabf89-2a6f-4fc3-b221-9df4dff819cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.320738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cabf89-2a6f-4fc3-b221-9df4dff819cb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.321017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cabf89-2a6f-4fc3-b221-9df4dff819cb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.321286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cabf89-2a6f-4fc3-b221-9df4dff819cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.332574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cabf89-2a6f-4fc3-b221-9df4dff819cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.334059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cabf89-2a6f-4fc3-b221-9df4dff819cb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.345831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-75cabf89-2a6f-4fc3-b221-9df4dff819cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:56.351112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:56.352209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:56.353188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:56.353188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:56.353188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:56.353188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:56.353188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 248/494] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:56.353588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2e059-4e5e-4220-8171-998c988e392e None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.353818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2e059-4e5e-4220-8171-998c988e392e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.354086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2e059-4e5e-4220-8171-998c988e392e None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.355337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 249/495] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:56.356221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2e059-4e5e-4220-8171-998c988e392e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.393052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f945234-aa11-4576-b948-37fc2401d3be None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.426337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225de080-56a8-4b02-8db0-53fb612fc482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.427014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225de080-56a8-4b02-8db0-53fb612fc482 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.427296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225de080-56a8-4b02-8db0-53fb612fc482 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.427554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225de080-56a8-4b02-8db0-53fb612fc482 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.428317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225de080-56a8-4b02-8db0-53fb612fc482 None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:56.428630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225de080-56a8-4b02-8db0-53fb612fc482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.430027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225de080-56a8-4b02-8db0-53fb612fc482 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.444058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 247/496] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:56.444058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a22798-ced5-436a-9b0c-c4c5a7ca40f2 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.444058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a22798-ced5-436a-9b0c-c4c5a7ca40f2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.444058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a22798-ced5-436a-9b0c-c4c5a7ca40f2 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.444058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 248/497] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:37:56.444058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a22798-ced5-436a-9b0c-c4c5a7ca40f2 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.458977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2e059-4e5e-4220-8171-998c988e392e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.478024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.491906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c6847-3ca4-419b-85f1-c6855a2610a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.492728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c6847-3ca4-419b-85f1-c6855a2610a0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.493039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c6847-3ca4-419b-85f1-c6855a2610a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.493306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c6847-3ca4-419b-85f1-c6855a2610a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.500879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c6847-3ca4-419b-85f1-c6855a2610a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.502389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c6847-3ca4-419b-85f1-c6855a2610a0 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-bb1c6847-3ca4-419b-85f1-c6855a2610a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:37:56.517846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:56.518764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:56.519716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 250/498] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:56.524839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4e69-c876-4c88-ba4e-712143be1902 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.525126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4e69-c876-4c88-ba4e-712143be1902 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.525404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4e69-c876-4c88-ba4e-712143be1902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.541694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-961b4e69-c876-4c88-ba4e-712143be1902 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:56.542926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-961b4e69-c876-4c88-ba4e-712143be1902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:56.544629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a22798-ced5-436a-9b0c-c4c5a7ca40f2 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.561717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.578285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.579010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.579251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.579519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.580269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:56.580600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.582004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.594128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 249/499] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:56.594916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.610656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 251/500] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:56.611931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b4dbd-3053-48ff-b4f2-f09bbdf8b9a7 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.612247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b4dbd-3053-48ff-b4f2-f09bbdf8b9a7 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.612519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b4dbd-3053-48ff-b4f2-f09bbdf8b9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.628621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-341b4dbd-3053-48ff-b4f2-f09bbdf8b9a7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:37:56.629778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-341b4dbd-3053-48ff-b4f2-f09bbdf8b9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:37:56.697666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 252/501] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:37:56.698407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b4dbd-3053-48ff-b4f2-f09bbdf8b9a7 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.702894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d04c608-513b-4234-8694-0d61a6e9fc85 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.737703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.738496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.739222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.739572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.740761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.751298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.752780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:56.770505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:56.771389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:56.772623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:56.772623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:56.772623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:56.772623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:56.772623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 250/502] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:56.772623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.799807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b4dbd-3053-48ff-b4f2-f09bbdf8b9a7 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.809609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.834436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.835101 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.835381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.835654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.836539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:56.836848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.838315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.874776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 253/503] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:56.875609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.882441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb7173-c6c7-4d04-aba1-ff38caa5ca0f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:56.915910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:56.916740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:56.917018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:56.917286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:56.924631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:56.926075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:37:56.941353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:56.942274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:56.943186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 251/504] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:56.943775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:56.973398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:56.976965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7665d8a-9805-4112-bc57-0d19409e01ad None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.010591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.011402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.011684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.011959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.022456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:37:57.030046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:57.030990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:57.031793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 254/505] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:57.032487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.033769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.044806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a6300-5556-4be0-a3e7-565652c36826 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.078752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.080224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.080511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.080783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.091314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.092777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:57.109312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:57.110205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:57.111071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:57.111071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:57.111071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:57.111071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:57.111071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 252/506] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:57.111333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.139284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a264e-7ff3-4e6f-ad4f-324c5f5d1d69 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.172472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.173152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.173419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.173678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.174471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:57.174767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.176187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.188464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 255/507] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:57.189250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.209001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.212810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e400e9d-6582-4c08-a674-e48b1f9dee01 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.246498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.247133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.247413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.247701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.259373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.293851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc037b-0750-4eb7-9550-e75337278351 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.321362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.322832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.328604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.329287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.329565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.329838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.330690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:57.330994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.332501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.349914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 256/508] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:57.350742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.425174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 253/509] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:57.426012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.430226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.452159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7891d0-ae7b-4b44-b28e-00025a227674 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.467252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.485687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.486370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.486650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.486924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.487698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:57.488037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.489443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.497648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 257/510] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:57.498439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.528021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f910e2-4300-4cd9-a22b-e8d0a94d001f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.561350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.561971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.562255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.562531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.563251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:57.563560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.564975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.575977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 254/511] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:57.576807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.600723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76cd81-b18f-4f4c-854d-c7d53efc8713 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.633599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.634740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.635069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.635311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.642993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.644474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.646200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:37:57.659757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:57.660868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 258/512] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:57.661782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.677725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235be503-69f6-4b22-aee0-68af1d58a821 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.679275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.712835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.713484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.713766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.714042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.760465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e03f9-59e0-48ce-af3a-ebbf1fdbd7f4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.779336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.780974 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.793847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.794654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.794936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.795204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.806210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.807660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:57.824642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:57.825670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:57.826661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:57.826661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:57.826661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:57.826661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:57.826661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 259/513] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:57.826661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.878033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.880295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 255/514] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:57.881083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:57.883302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:57.934592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77a90d-eada-4757-a341-68e8f45efe5f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.967496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:57.968351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:57.968631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:57.968902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:57.978717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeecd1-3e6f-4f57-bf2b-e5cb4e0660cd None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:57.979179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:57.980734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:58.000289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:58.001233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:58.002376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:58.002376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:58.002376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:58.002376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:58.002376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 260/515] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:58.002376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.015459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.016165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.016450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.016724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.017442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.018904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.044211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 256/516] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:37:57 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:37:58.045188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.114864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.117398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293bf1-0f2a-4e28-868c-4365667edfb7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.119686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.157574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.158282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.158579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.158836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.159699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:58.160035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.161570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.168491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa9e83-2bd1-4646-a3f5-a80f61fa9de4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.181983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 261/517] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:58.182868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.213464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.214265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.214575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.214757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.215598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:58.215909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.217398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.232957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 257/518] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:58.235871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.313088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedf2f5-f2a4-49d9-aaac-e419019cab1b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.344644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c555a6f-aea2-4376-bf7c-c703bf6170da None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.350216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.351043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.351378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.351658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.362345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.364785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.366661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.379781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.380477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.380767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.381042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.381772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:58.382078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.383460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:58.387417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:58.388584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:58.389451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:58.389451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:58.389451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:58.389451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:58.392364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 262/519] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:58.393804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.400340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 258/520] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:58.400340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.409436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.495338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408ae4-e71d-4cb4-b920-c248b6470e9b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.504539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62cfde6-e4be-42a5-abdf-e83d87b77250 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.533507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.534166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.534487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.534749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.541934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.542769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.543046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.543313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.550406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.551884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:37:58.567219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:58.568141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:58.569034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 259/521] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:58.569524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.609252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.614198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.615694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.630240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.675926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0750670-c89d-4a4b-93c7-355890b9b7a5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.711115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.711780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.712093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.712369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.713093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:58.713398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.714988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.730768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 260/522] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:58.730768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.730996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 263/523] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:58.730996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.830536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8ba85-77f3-4e44-8079-11df62299f03 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.833299 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.837409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3282f3-f866-4016-9b55-10179eab2813 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:58.845587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:58.868018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.868676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.868960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.869235 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.870028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:58.870353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.871755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:58.871882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.872613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:58.872892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:58.873166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:58.881084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:58.882455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:58.896085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 264/524] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:58.896085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:37:58.896764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:58.897781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:58.898687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 261/525] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:58.899244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.001097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810b842-d83a-46a3-87ed-429200df2b86 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.003799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6fc74-9053-4081-971c-511ccbdc14ab None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.038367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.039148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.039430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.039699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.041923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.043043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.043324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.043596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.046501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.048012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.055412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.056883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:37:59.062258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:59.063245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:37:59.064138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 265/526] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:59.064628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:37:59.075375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:59.076316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:59.077169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:37:59.077169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:37:59.077169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:37:59.077169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:59.077169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 262/527] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:59.077947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.095806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:59.101263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:59.175078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5893b-e473-40ea-a40a-e2d2a238d936 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.196971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebcc50-2cf3-4702-9841-226d6165a9ec None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.212720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.213379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.213658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.213930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.214888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=4da1612f1c1740ceb2a918f0a8dd057b) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:59.215199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.216682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.232018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 266/528] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/role_assignments?user.id=4da1612f1c1740ceb2a918f0a8dd057b&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:59.232018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.243491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.244397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.244770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.245110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.246118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:59.246496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.248368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.305533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 263/529] 10.222.0.50 () {56 vars in 1139 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:59.306617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.361047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:59.371747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0f95-16cf-429c-8a10-615770396100 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.383529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:59.414175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.415001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.415338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.415715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.416769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:59.417256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.419240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.427717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158204b-d5f8-4718-b50b-9e7e6296696f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.435127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 267/530] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:59.436222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.466928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.467685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.468081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.468417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.469212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:59.469576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.471013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.488565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 264/531] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:59.488565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.551031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e993b7-436e-4dff-89df-00bfed21547f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.586190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.586884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.587094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.587366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.606559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd00ea-1063-4085-9944-09fc43e48e65 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.612679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:59.639610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:59.647927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.648862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.649343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.650030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.651239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=7c9cdda3d7884298a791f90523f3ab91) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:37:59.651808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.654019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.661432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 265/532] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/role_assignments?user.id=7c9cdda3d7884298a791f90523f3ab91&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:37:59.662784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.672106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.673787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.802878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb8cd0-2f09-4701-ae50-e81c50b5d411 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:37:59.823576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 268/533] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:37:59.828890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.882279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:37:59.883094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:37:59.883376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:37:59.883648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:37:59.895479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:37:59.896999 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:37:59.912866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:37:59.913799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:37:59.915096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 266/534] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:37:59.920686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:37:59.952341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:37:59.982857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:00.038008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec613642-a3a8-439f-a7c7-b2d09c1bf162 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.076143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.076737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.077022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.077457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.078613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:00.078941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:00.080440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:00.098230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccd34b-1039-4443-acbf-271e84978d7d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.104477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 267/535] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:00.106227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.181266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d0423-bf70-4b58-95da-8bf1f942dc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.182241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d0423-bf70-4b58-95da-8bf1f942dc37 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.182519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d0423-bf70-4b58-95da-8bf1f942dc37 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.183305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d0423-bf70-4b58-95da-8bf1f942dc37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.195561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d0423-bf70-4b58-95da-8bf1f942dc37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:00.197083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d0423-bf70-4b58-95da-8bf1f942dc37 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-811d0423-bf70-4b58-95da-8bf1f942dc37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:00.219189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:00.220118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:00.221440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:00.221440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:00.221440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:00.221440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:00.221440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 269/536] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:00.221725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b12a8-a7bd-4168-9fab-44556131bbf2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.223752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69301d89-169d-4cf7-b5cf-2e387e6d8df9 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.223752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69301d89-169d-4cf7-b5cf-2e387e6d8df9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.223752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69301d89-169d-4cf7-b5cf-2e387e6d8df9 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.228320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 270/537] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:00.229975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69301d89-169d-4cf7-b5cf-2e387e6d8df9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.243047 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:00.273449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f35b8-e983-4006-96bd-824a0d431024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.274518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f35b8-e983-4006-96bd-824a0d431024 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.274941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f35b8-e983-4006-96bd-824a0d431024 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.275324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f35b8-e983-4006-96bd-824a0d431024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.276206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f35b8-e983-4006-96bd-824a0d431024 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=e4cb4d5213c647459e1c21c311a4a305) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:00.276509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f35b8-e983-4006-96bd-824a0d431024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:00.278463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f35b8-e983-4006-96bd-824a0d431024 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:00.289421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 268/538] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/role_assignments?user.id=e4cb4d5213c647459e1c21c311a4a305&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:00.290795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1705b3-4d03-46f5-827a-3c27ef1db616 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.291558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1705b3-4d03-46f5-827a-3c27ef1db616 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.291837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1705b3-4d03-46f5-827a-3c27ef1db616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.315472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3e1705b3-4d03-46f5-827a-3c27ef1db616 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:00.316846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3e1705b3-4d03-46f5-827a-3c27ef1db616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:00.366744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69301d89-169d-4cf7-b5cf-2e387e6d8df9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.378160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:00.389142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 269/539] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:00.407957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.409152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.409670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.410132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.411054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:00.411635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:00.413229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1705b3-4d03-46f5-827a-3c27ef1db616 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.413570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:00.427326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 271/540] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:00.433479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.519025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1705b3-4d03-46f5-827a-3c27ef1db616 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.527289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:00.550810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ed2f8-cc45-44a9-973c-773655f9dc25 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.555626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.556503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.556783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.557066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.569434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:00.585787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:00.586789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:00.587585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 270/541] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:00.598700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.606221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.607328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.608115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.608622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.618002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:00.619922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:00.637824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:00.638748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:00.639785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:00.642471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 272/542] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:00.648256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.692374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:00.712371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807538b-f6fd-45c2-805b-958c1ec520dc None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.755212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053ad3d7-579a-4147-a40c-44449caf13d7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.780849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.781529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.781811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.782085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.782849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:00.783144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:00.784680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:00.796857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.797898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:00.798367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:00.798826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:00.800668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:00.803324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:00.805028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:00.805480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 271/543] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:00.808416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.828115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 273/544] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:00.834176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:00.870752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:00.940429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:00.956428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cb583-24a5-476b-aef0-b356372b3f9c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:00.997800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:00.999568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.000291 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.002250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.004596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=5e17d25a83fe48d99ddbd209fbea8486) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:01.005008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:01.007364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:01.022647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 274/545] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/role_assignments?user.id=5e17d25a83fe48d99ddbd209fbea8486&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:01.028079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951f3c2-70dd-4e94-8c98-4959098cce38 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:01.073950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:01.075032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.075403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.075813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.092587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:01.094255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:01.119034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:01.120141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:01.121265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:01.121265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:01.121265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:01.121265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:01.122885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 272/546] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:01.128410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:01.271846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c7532-a9f8-4acd-b2ff-32edc089c479 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:01.313911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:01.314710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.314994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.315269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.316009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:01.316316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:01.321049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:01.337872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 275/547] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:01.343260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:01.441839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:01.468174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51e959-9d67-4b40-b87b-e53e41843235 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:01.504237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:01.505037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.505313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.505585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.514253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:01.515833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:01.530429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:01.531340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:01.532853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 273/548] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:01.542146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:01.566827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:01.667246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c77257-31ba-477e-bcc6-b26c224d3c35 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:01.702760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bb196-48a4-44da-a9ea-d102d73bde84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:01.703366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bb196-48a4-44da-a9ea-d102d73bde84 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.703644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bb196-48a4-44da-a9ea-d102d73bde84 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.703921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bb196-48a4-44da-a9ea-d102d73bde84 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.704796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bb196-48a4-44da-a9ea-d102d73bde84 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:01.705107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bb196-48a4-44da-a9ea-d102d73bde84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:01.706560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bb196-48a4-44da-a9ea-d102d73bde84 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:01.733906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 276/549] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Jun 30 15:38:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:01.739835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:01.879843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769674f-89bf-4517-a777-4213e5737e54 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:01.916559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971fad6-45b7-4068-aa62-5d468dd4b210 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.916916 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971fad6-45b7-4068-aa62-5d468dd4b210 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.917112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971fad6-45b7-4068-aa62-5d468dd4b210 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.918560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 277/550] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:01.923713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef9bca-fbfe-4a19-a04a-655c63d9e388 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.924011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef9bca-fbfe-4a19-a04a-655c63d9e388 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.924309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ef9bca-fbfe-4a19-a04a-655c63d9e388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.926102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:01.926756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:01.927035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:01.927305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:01.940323 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:01.952622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-89ef9bca-fbfe-4a19-a04a-655c63d9e388 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:01.954318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-89ef9bca-fbfe-4a19-a04a-655c63d9e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:02.017736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:02.019222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:02.027673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.033952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 278/551] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:38:02.036015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207632ed-6789-4319-9e75-9c416fd41693 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.036315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207632ed-6789-4319-9e75-9c416fd41693 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.036649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207632ed-6789-4319-9e75-9c416fd41693 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.042107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 279/552] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:02.043851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78857-2faa-49ec-9178-6c6493b6df80 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.044382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78857-2faa-49ec-9178-6c6493b6df80 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.044943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f78857-2faa-49ec-9178-6c6493b6df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.061553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-71f78857-2faa-49ec-9178-6c6493b6df80 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:02.063112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-71f78857-2faa-49ec-9178-6c6493b6df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:02.144010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 280/553] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:38:02.144010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f78857-2faa-49ec-9178-6c6493b6df80 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.146627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 274/554] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:02.151660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.208190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.244761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f78857-2faa-49ec-9178-6c6493b6df80 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.272221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316966-7e3e-4228-aaf4-f7474612ec64 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.276094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.282510 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.283324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.283597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.283865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.297840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:02.306311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:02.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 281/555] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:02.308919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.324198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.325415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.325986 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.326525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.338108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:02.340144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:02.359314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:02.360644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:02.361524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:02.361524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:02.361524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:02.361524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:02.363752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 275/556] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:02.364954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.413725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49692f89-4ba6-4ce9-bf14-0fe2c78bebdb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.449663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.450768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.450881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.451085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.462153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:02.465348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:02.472415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:02.473426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:02.475280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:02.478596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 282/557] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:02.479489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.480781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d0e0f-fdd4-47ed-ae87-cf627e9fdcc9 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.520462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.521599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.522099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.522697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.523646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:02.524219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:02.525990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:02.533710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.537542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 276/558] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:02.538826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.589031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fbe2f-ea45-4d29-9169-139423440256 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.624684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.625295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.625574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.625842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.626560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:02.626864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:02.628381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:02.642080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 283/559] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:02.642991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.657336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59110f7e-1a7f-4520-b31c-0efeb827280e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.710093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.710782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.711066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.711341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.712147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:02.712516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:02.713873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:02.722428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 277/560] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:02.723153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.741581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.756086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76242852-9101-4bab-bcb4-858629e058b5 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.767674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.794907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43275e-828b-4747-9025-f0c4dc50f805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.795710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43275e-828b-4747-9025-f0c4dc50f805 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.796624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43275e-828b-4747-9025-f0c4dc50f805 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.796624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43275e-828b-4747-9025-f0c4dc50f805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.806733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43275e-828b-4747-9025-f0c4dc50f805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:02.808264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43275e-828b-4747-9025-f0c4dc50f805 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-aa43275e-828b-4747-9025-f0c4dc50f805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:02.826210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:02.827121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:02.828627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:02.828627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:02.828627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:02.828627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:02.828898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb56b-c534-4518-80f4-67b9a56f4b48 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.832206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 284/561] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:02.833362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090c9b-507d-413b-bd90-a297328464d4 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.833642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090c9b-507d-413b-bd90-a297328464d4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.833911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090c9b-507d-413b-bd90-a297328464d4 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.835331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 285/562] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:02.835855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090c9b-507d-413b-bd90-a297328464d4 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.866616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.867409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.867684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.868023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:02.874927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:02.876397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:02.891137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:02.892070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:02.892980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 278/563] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:02.893474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:02.946383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090c9b-507d-413b-bd90-a297328464d4 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:02.978493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:02.986113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fb3ef-97af-4ff6-bb71-daf9c9e9257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:02.987840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb3ef-97af-4ff6-bb71-daf9c9e9257f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:02.988346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb3ef-97af-4ff6-bb71-daf9c9e9257f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:02.988835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fb3ef-97af-4ff6-bb71-daf9c9e9257f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.001648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fb3ef-97af-4ff6-bb71-daf9c9e9257f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.003222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fb3ef-97af-4ff6-bb71-daf9c9e9257f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.004231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b08dcd-c50e-4e53-937f-dec4b3d9fb6a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-8f8fb3ef-97af-4ff6-bb71-daf9c9e9257f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:03.021602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:03.022595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:03.023475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:03.023475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:03.023475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:03.023475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:03.023475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 286/564] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:03.024217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3758-4436-4168-a437-a17240bb8eb5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.024315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3758-4436-4168-a437-a17240bb8eb5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.024591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff3758-4436-4168-a437-a17240bb8eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.043150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c0ff3758-4436-4168-a437-a17240bb8eb5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:03.044361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c0ff3758-4436-4168-a437-a17240bb8eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:03.048189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.048863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.049121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.049420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.050152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:03.050457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.051971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.054046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:03.066455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 279/565] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:03.067535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.119063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 287/566] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:03.120016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3758-4436-4168-a437-a17240bb8eb5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.178623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c1cb6-0991-47ea-9cb1-1967c12e31fb None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.213791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.214591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.214875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.215156 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.215929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:03.216266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.217724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.220560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:03.233698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 280/567] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:03.234445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.240783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff3758-4436-4168-a437-a17240bb8eb5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.281112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.281963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.282348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.282722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.283557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:03.283962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.285481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.298660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 288/568] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:03.299605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.304092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:03.353188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac451cd7-934a-4f70-ae8e-5fc0f6629665 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.389653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.390940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.391493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.391956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.400071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.401873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.401985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2694c2b-e6fd-4ce2-8891-e77d1dbc3075 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:03.417651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:03.418895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:38:03.420103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:03.423977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 281/569] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:03.424928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.438484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.439290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.439690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.439835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.450806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:03.458624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:03.459664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 289/570] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:03.461279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.497916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:03.527849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:03.541529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8dfd-55d6-40c8-b339-6cb348f1e779 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.564816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925edf51-2fc0-49ab-972d-7e9913887eb7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.576669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.577579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.577953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.578415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.579666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2933f88027fc435eba3064124e2dd5d5) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:03.580134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.582587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.589610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 282/571] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/role_assignments?user.id=2933f88027fc435eba3064124e2dd5d5&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:03.590426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.603895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.604827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.605167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.605459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.614525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.616024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:03.631865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:03.632987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 290/572] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:03.633970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.722941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c9cf5-4aa7-4175-8f78-9e4a81ca533a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.745734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:03.750251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71750410-7db0-4e5a-9c51-4c6b5258f4cc None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.761784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e7872-4057-41ba-96c0-adb0d391aaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.762464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e7872-4057-41ba-96c0-adb0d391aaa3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.762742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e7872-4057-41ba-96c0-adb0d391aaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.763013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e7872-4057-41ba-96c0-adb0d391aaa3 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.763860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e7872-4057-41ba-96c0-adb0d391aaa3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:03.764206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e7872-4057-41ba-96c0-adb0d391aaa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.765656 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e7872-4057-41ba-96c0-adb0d391aaa3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.782883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 283/573] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:03.784613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:03.786778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.787594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.787917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.788274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.788818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edf304-f39b-4ea8-b0fe-3f29b3329ac3 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.789075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:03.789180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edf304-f39b-4ea8-b0fe-3f29b3329ac3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.789367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edf304-f39b-4ea8-b0fe-3f29b3329ac3 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.789474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.792157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 284/574] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:03.792300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.792439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edf304-f39b-4ea8-b0fe-3f29b3329ac3 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.801974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 291/575] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:03.802987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:03.900131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edf304-f39b-4ea8-b0fe-3f29b3329ac3 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.910196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2130722-b5b3-4c48-84d0-647699a819ae None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:03.936793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0114841-1d73-4425-9c84-1954d10bf8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.937734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0114841-1d73-4425-9c84-1954d10bf8ca None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.938182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0114841-1d73-4425-9c84-1954d10bf8ca None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.938646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0114841-1d73-4425-9c84-1954d10bf8ca None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.939751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0114841-1d73-4425-9c84-1954d10bf8ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:03.940086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0114841-1d73-4425-9c84-1954d10bf8ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.941533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0114841-1d73-4425-9c84-1954d10bf8ca None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.944029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:03.945014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.945297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.945573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.956281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:03.957736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:03.960344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 285/576] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:03.961757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc0e4-89e3-4803-9daf-083d2d935d36 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:03.962227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc0e4-89e3-4803-9daf-083d2d935d36 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:03.962677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddc0e4-89e3-4803-9daf-083d2d935d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:03.979034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f1ddc0e4-89e3-4803-9daf-083d2d935d36 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:03.980342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:03.981210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:03.986733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:03.986733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:03.986733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:03.986733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:03.986733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 292/577] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:03.987045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f1ddc0e4-89e3-4803-9daf-083d2d935d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:03.987123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.016452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.027005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.063490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 286/578] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:04.064381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddc0e4-89e3-4803-9daf-083d2d935d36 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.093224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71fc0-6e0e-4264-ab56-fd64b60237fb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.128610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.129418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.129703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.130378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.142915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.144410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:04.163479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:04.164622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:04.170794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:04.170794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:04.170794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:04.170794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:04.170794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 293/579] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:04.170794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.182029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddc0e4-89e3-4803-9daf-083d2d935d36 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.217423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.218223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.218503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.218772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.229771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.231280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:04.249257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:04.250200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:04.251139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:04.251139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:04.251139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:04.251139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:04.251139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 287/580] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:04.251415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.268079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.278276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb6b00-2f55-4ebb-96c4-b4b199b0744e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.278479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.325000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.325675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.325950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.326218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.326950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:04.327665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.328992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.342550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 294/581] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:04.347585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.376751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265e76b-8277-467b-8f92-02053ddd9bfe None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.413314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.414118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.414400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.414673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.427962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:04.437764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:04.438711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:04.439578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:04.440256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 288/582] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:04.440813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.452212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e420e07-0be9-4d93-af9f-82daeafea5b0 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.488554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.489429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.489887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.490308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.491170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:04.491617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.493194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.505036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 295/583] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:04.506015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.520910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.527959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.550663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720aa69-9811-4f34-afb4-b86f8b470681 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.584909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.585572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.585856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.586127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.586854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:04.587162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.588593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.600855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 289/584] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:04.602040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.616326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6812dd0-2631-40a6-ab1e-7927b3a1b53a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.655715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.656554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.656839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.657117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.664558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.666250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:04.682531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:04.683631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 296/585] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:04.684965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.713548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a379b4b-a465-4e78-9bc2-a265bb201e8f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.747790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.748480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.748760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.749030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.749774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:04.750078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.751465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.756713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.764689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:04.765192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 290/586] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:04.766108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.797748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b78a6-ed06-4c50-90a6-417e045b6102 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.834216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.835031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.835313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.835586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.842987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.844463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:04.858694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:04.859658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 297/587] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:04.865617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.880221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8965e2fd-77e8-487f-b390-5d223ba77336 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:04.916573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:04.917468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:04.917756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:04.918039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:04.924995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:04.926432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:04.940873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:04.941781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:04.942687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 291/588] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:04.943877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:04.968971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43793778-f0cb-4ef8-b75b-129e9b671d11 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.001529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.004111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.008965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.009986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.010232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.010565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.011384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:05.011728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.013095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.029503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 298/589] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:05.029503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.053433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe0547-c06f-4e3d-98d7-74b229a095b2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.090157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.090968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.091243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.091514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.102186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.103734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:05.122034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:05.123167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:05.124298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:05.124298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:05.124298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:05.124298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:05.124298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 292/590] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:05.124610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.145344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1be45-20da-4732-adf2-e839eadb6c17 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.179940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.180638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.180920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.181187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.182015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:05.182316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.184245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.207663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 299/591] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:05.207663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.232011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5042-61fa-4bf5-b269-5617976a8895 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.246067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.260356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.274575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.275235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.275511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.275835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.276808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:05.277258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.278552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.293233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 293/592] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:05.294061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.322757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f650ed-f1e2-4427-830c-770fc2df40aa None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.357626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.358524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.358806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.359079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/7c9cdda3d7884298a791f90523f3ab91/roles/fbd9e52b6be84e29999adebc31c75d72` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.373842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=7c9cdda3d7884298a791f90523f3ab91, role_id=fbd9e52b6be84e29999adebc31c75d72)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.375294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.400025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 300/593] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:05 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/7c9cdda3d7884298a791f90523f3ab91/roles/fbd9e52b6be84e29999adebc31c75d72 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:05.400025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.406261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf20b-f779-4587-a037-665695dad3fb None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.445261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.445937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.446218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.446491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.447346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:05.447655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.449198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.472398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 294/594] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:05.473628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.510752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.512243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.516592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f542f-1909-4eef-8937-baab1f21097e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.556305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.557586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.558124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.558597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.571470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.574075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:05.597406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:05.598353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:05.599245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:05.599245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:05.599245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:05.599245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:05.599919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bbc24-ee9e-46f6-9560-4f6b5aeb10ad None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.602138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 301/595] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:05.603095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.642133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb14e2f9-353d-4a28-b7e5-db9dc80bd17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.643021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14e2f9-353d-4a28-b7e5-db9dc80bd17d None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.643298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14e2f9-353d-4a28-b7e5-db9dc80bd17d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.643570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb14e2f9-353d-4a28-b7e5-db9dc80bd17d None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/4da1612f1c1740ceb2a918f0a8dd057b/roles/fbd9e52b6be84e29999adebc31c75d72` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.664429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14e2f9-353d-4a28-b7e5-db9dc80bd17d None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=4da1612f1c1740ceb2a918f0a8dd057b, role_id=fbd9e52b6be84e29999adebc31c75d72)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.666211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb14e2f9-353d-4a28-b7e5-db9dc80bd17d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.681121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 295/596] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:05 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/4da1612f1c1740ceb2a918f0a8dd057b/roles/fbd9e52b6be84e29999adebc31c75d72 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:05.684909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d553-c646-42a0-bae6-b926c9f2a701 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.685404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d553-c646-42a0-bae6-b926c9f2a701 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.685924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785d553-c646-42a0-bae6-b926c9f2a701 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.687555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 296/597] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:05.688621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785d553-c646-42a0-bae6-b926c9f2a701 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.729165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a03361-c5c9-412e-a0e7-661ce23df81c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.768515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08960fa9-abf5-4cee-b81b-7ab5c7d80732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.769525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08960fa9-abf5-4cee-b81b-7ab5c7d80732 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.769850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08960fa9-abf5-4cee-b81b-7ab5c7d80732 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.770188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08960fa9-abf5-4cee-b81b-7ab5c7d80732 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.779172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08960fa9-abf5-4cee-b81b-7ab5c7d80732 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.781019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08960fa9-abf5-4cee-b81b-7ab5c7d80732 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.791355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-08960fa9-abf5-4cee-b81b-7ab5c7d80732 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:05.799451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:05.800594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 15:38:05.801544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:05.805474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 302/598] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:05.807126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd441a-34dc-4633-87ed-8912f1482a35 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.807560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd441a-34dc-4633-87ed-8912f1482a35 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.808055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd441a-34dc-4633-87ed-8912f1482a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.822334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785d553-c646-42a0-bae6-b926c9f2a701 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:05.834693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a9dd441a-34dc-4633-87ed-8912f1482a35 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:05.836413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a9dd441a-34dc-4633-87ed-8912f1482a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:05.840280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:05.859894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:05.860587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:05.860895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:05.861133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:05.861868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:05.862170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:05.863677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:05.880081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 297/599] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:05.881830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:05.926105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 303/600] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:05.932012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd441a-34dc-4633-87ed-8912f1482a35 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.008172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d433a72-fafa-4e36-921a-557287e30ad8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.050837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.051516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.051795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.052237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.053388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:06.053700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.055565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.062527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd441a-34dc-4633-87ed-8912f1482a35 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.073559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:06.079970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 298/601] 10.222.0.50 () {56 vars in 1139 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:06.081368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.101283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.102197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.102571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.102941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.110573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.112315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.115275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:06.129410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:06.130535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:06.131645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:06.138437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 304/602] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:06.138437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.210268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f36965-1cc4-4b34-876b-2cc2c359d2d8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.246800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb4a5a-55dd-410b-a3dc-de2c34e30c80 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.255816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633496-b239-42a9-970a-953035f4acad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.257400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633496-b239-42a9-970a-953035f4acad None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.257823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633496-b239-42a9-970a-953035f4acad None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.258624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633496-b239-42a9-970a-953035f4acad None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.273154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f633496-b239-42a9-970a-953035f4acad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5f633496-b239-42a9-970a-953035f4acad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:06.284009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:06.285147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:06.286201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:06.288613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.288791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 299/603] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:06.289512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.289886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.290077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633496-b239-42a9-970a-953035f4acad None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.290335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.303713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.305427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:06.324538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:06.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:06.327094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:06.327094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:06.327094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:06.327094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:06.329129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 305/604] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:06.330904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.370032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:06.393618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:06.425668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633496-b239-42a9-970a-953035f4acad None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.446382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8040acd-ace7-43cb-a16a-c381336b9b54 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.469600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.470247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.470525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.470795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.471583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:06.471892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.473371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.488560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.489236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.489513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.489782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.490534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:06.490838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.496412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.496532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 300/605] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:06.496949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.505257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 306/606] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:06.506058 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.611965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c7640-0213-4986-831d-3d3a21ef3457 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.612523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d42c9d-cd9b-4502-b71e-27ee2fd99257 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.626371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:06.634935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:06.649355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.650260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.650554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.650804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/e4cb4d5213c647459e1c21c311a4a305/roles/fbd9e52b6be84e29999adebc31c75d72` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.652328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.653164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.653640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.654141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.655007 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:06.655470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.657020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.666341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=e4cb4d5213c647459e1c21c311a4a305, role_id=fbd9e52b6be84e29999adebc31c75d72)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.667698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.670051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 301/607] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:06.671130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.688226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 307/608] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:06 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/e4cb4d5213c647459e1c21c311a4a305/roles/fbd9e52b6be84e29999adebc31c75d72 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:06.689253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.781034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a66c-543a-4176-b83a-24bf08bf0925 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.806167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44604478-0c0d-4b4e-b23d-1b66fe83b003 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:06.819095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.820204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.820776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.821272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.833697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.835845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.844066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:06.844835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:06.845155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:06.845436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:06.852957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:06.854523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:06.857072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:06.857964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:06.858840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:06.858840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:06.858840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:06.858840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:06.861195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 302/609] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:06.867147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:06.869702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:06.870709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:06.871615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 308/610] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:06.877196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:06.913608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:06.945609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:07.002684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb77f22-14aa-4ed4-a031-565c112fe6d2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:07.036976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d44f81-bfda-4cb6-b7ea-4ad90f4aca9a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:07.046781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:07.048215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.048792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.049282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.050162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:07.050732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:07.052554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:07.069721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 303/611] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:07.077049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:07.082533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:07.084582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.087652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.088187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.089182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:07.089700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:07.091297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:07.115212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 309/612] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:07.121011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:07.238257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:07.249869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f61163-e9f5-4d83-9611-90d02f7ca022 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:07.292023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:07.295920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:07.296760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.297043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.297319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.317162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b5ea7-aa8e-4416-ae92-b6abbddb286d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:07.319174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:07.320675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:07.343837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:07.344905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:07.345870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 304/613] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:07.351101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:07.388993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:07.389905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.390187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.390459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/5e17d25a83fe48d99ddbd209fbea8486/roles/fbd9e52b6be84e29999adebc31c75d72` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.412582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=5e17d25a83fe48d99ddbd209fbea8486, role_id=fbd9e52b6be84e29999adebc31c75d72)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:07.414158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:07.431738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 310/614] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:07 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/5e17d25a83fe48d99ddbd209fbea8486/roles/fbd9e52b6be84e29999adebc31c75d72 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:07.489529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05d8d-9270-400d-b5dc-9ed3b0ddbaf6 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:07.527837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604078c0-920d-4626-8554-3805de42a740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:07.528559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604078c0-920d-4626-8554-3805de42a740 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.528836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604078c0-920d-4626-8554-3805de42a740 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.529136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604078c0-920d-4626-8554-3805de42a740 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.530104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604078c0-920d-4626-8554-3805de42a740 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:07.530412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604078c0-920d-4626-8554-3805de42a740 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:07.531881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604078c0-920d-4626-8554-3805de42a740 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:07.552146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 305/615] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Jun 30 15:38:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:07.558216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:07.648012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330d65-4414-4a24-9383-40fe9e102d52 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.648556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330d65-4414-4a24-9383-40fe9e102d52 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.648734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c330d65-4414-4a24-9383-40fe9e102d52 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.650368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 306/616] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:07.655895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a9c6-59ee-47c4-9114-3bffa0f2dc3c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.656218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a9c6-59ee-47c4-9114-3bffa0f2dc3c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.656495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829a9c6-59ee-47c4-9114-3bffa0f2dc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.684175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1829a9c6-59ee-47c4-9114-3bffa0f2dc3c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:07.685432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1829a9c6-59ee-47c4-9114-3bffa0f2dc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:07.692749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5c3d5-8f09-4758-b98c-a6e4fda13d71 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:07.693324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:07.740154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:07.741355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.743516 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.744042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:07.749059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:07.771457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:07.773316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:07.806193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:07.807582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:07.808552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:07.808552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:07.808552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:07.808552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:07.811073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 311/617] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:07.816042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 307/618] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:07.817141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:07.879558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a9c6-59ee-47c4-9114-3bffa0f2dc3c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:07.945500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2919ffe-fe81-41fc-973e-62010275ccfb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:07.993804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:07.997647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:07.998153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:07.998652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.000403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:08.001506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.003288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.029989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 312/619] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:08.033434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.074777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829a9c6-59ee-47c4-9114-3bffa0f2dc3c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.092415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:08.128794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02299213-443c-4e3a-a1d5-62cb7c7273ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.129849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02299213-443c-4e3a-a1d5-62cb7c7273ec None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.130148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02299213-443c-4e3a-a1d5-62cb7c7273ec None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.130421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02299213-443c-4e3a-a1d5-62cb7c7273ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.141917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:08.146199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02299213-443c-4e3a-a1d5-62cb7c7273ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:08.155088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499310e9-3492-44e6-b2e5-4a15c52ffa01 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-02299213-443c-4e3a-a1d5-62cb7c7273ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:08.156787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:08.157905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:08.158789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:08.161766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 308/620] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:08.163435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db072386-0159-48a3-bd3d-c9f885da89cf None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.163959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db072386-0159-48a3-bd3d-c9f885da89cf None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.164470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db072386-0159-48a3-bd3d-c9f885da89cf None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.166504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 309/621] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:08.169219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db072386-0159-48a3-bd3d-c9f885da89cf None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.198701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb559f0-a753-4e62-865c-bdfa955e63a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.199777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb559f0-a753-4e62-865c-bdfa955e63a7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.200270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb559f0-a753-4e62-865c-bdfa955e63a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.200642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb559f0-a753-4e62-865c-bdfa955e63a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.209096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb559f0-a753-4e62-865c-bdfa955e63a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.210768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb559f0-a753-4e62-865c-bdfa955e63a7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-fdb559f0-a753-4e62-865c-bdfa955e63a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:08.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:08.227559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:08.228720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:08.230440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 313/622] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:08.231951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769aee-0811-4fbc-8540-5b201cbefaa2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.232336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769aee-0811-4fbc-8540-5b201cbefaa2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.232856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769aee-0811-4fbc-8540-5b201cbefaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.251305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c4769aee-0811-4fbc-8540-5b201cbefaa2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:08.252625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c4769aee-0811-4fbc-8540-5b201cbefaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:08.312014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db072386-0159-48a3-bd3d-c9f885da89cf None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.330627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 314/623] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:08.331726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4769aee-0811-4fbc-8540-5b201cbefaa2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.363498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.365161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.365505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.365935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.366834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:08.367237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.368805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.377877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:08.385117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 310/624] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:08.386274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.450865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:08.464196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4769aee-0811-4fbc-8540-5b201cbefaa2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.499978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.500975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.501249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31023-ac6a-4f91-b272-8f06c01fbfc8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.501480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.501795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.502671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:08.503072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.504898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.518726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 315/625] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:08.519761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.543370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.545463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.545893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.546307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.560910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:08.571525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:08.572701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:08.573832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:08.581981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 311/626] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:08.581981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.636168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:08.645421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b9c06-2763-4e90-8d9b-dbf2496b44fd None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.689560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105af390-f424-452d-ba6e-1ee4e694f2af None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.696359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.697441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.697945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.698867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.712680 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.714624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.716136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:08.734522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:08.735350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:08.736308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:08.737199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:08.737199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:08.737199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:08.737199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:08.737519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.737519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.737519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2933f88027fc435eba3064124e2dd5d5/roles/fbd9e52b6be84e29999adebc31c75d72` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.739877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 316/627] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:08.741037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.761569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=2933f88027fc435eba3064124e2dd5d5, role_id=fbd9e52b6be84e29999adebc31c75d72)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.763070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.781197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 312/628] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:08 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2933f88027fc435eba3064124e2dd5d5/roles/fbd9e52b6be84e29999adebc31c75d72 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:08.782240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.864876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bbeab-8ae0-4d25-ba82-42bf829152ae None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.908254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.909187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.909534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.909878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.911001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:08.911380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.913748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.921453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8883c9-0963-449a-b9b3-9a88aa4360e3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:08.925566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 317/629] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:08.937189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:08.957133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:08.965953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:08.967048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:08.967601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:08.968102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:08.969164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:08.970542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:08.972616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:08.987852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 313/630] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:08.994062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.038945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:09.092292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1196f3-cc89-465a-91ac-ad2e57ee351b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.131780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.132854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.133362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.133866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.135097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c1ee8-c044-4b26-b8ca-116cb7f034cf None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.147736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:09.160012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:09.178495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.179316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.179594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.179865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.187983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:09.189484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:09.200790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:09.201874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:09.202883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:09.202883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:09.202883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:09.202883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:09.204404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:09.205336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:09.206331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 314/631] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:09.207526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 318/632] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:09.211295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.325936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:09.348119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:09.359281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a66a4c-129a-466f-a12c-61d0d3ed2801 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.374283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4524b7a-807e-4cef-bee8-421c4c46f351 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.396997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b71035-3e7e-4c8f-9403-338580d51b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.398149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b71035-3e7e-4c8f-9403-338580d51b2c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.398769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b71035-3e7e-4c8f-9403-338580d51b2c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.399281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b71035-3e7e-4c8f-9403-338580d51b2c None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.400444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b71035-3e7e-4c8f-9403-338580d51b2c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:09.401054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b71035-3e7e-4c8f-9403-338580d51b2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:09.402858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b71035-3e7e-4c8f-9403-338580d51b2c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:09.412341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.413016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.413296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.413567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.414789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:09.414789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:09.415978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:09.426199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 315/633] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Jun 30 15:38:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:09.427938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0f2a-add5-483e-a61d-4246a3f7f8f6 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.428526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0f2a-add5-483e-a61d-4246a3f7f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.429025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0f2a-add5-483e-a61d-4246a3f7f8f6 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.434351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 319/635] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:09.434594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 316/635] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:09.435480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.436749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce06ef-55e4-465a-ad7b-4485a0f60d51 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.437255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce06ef-55e4-465a-ad7b-4485a0f60d51 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.437725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce06ef-55e4-465a-ad7b-4485a0f60d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.455383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-11ce06ef-55e4-465a-ad7b-4485a0f60d51 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:09.456929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-11ce06ef-55e4-465a-ad7b-4485a0f60d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:09.534238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 317/636] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:38:09.535526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce06ef-55e4-465a-ad7b-4485a0f60d51 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.536475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6005b-f42f-47b6-9d32-ed88695857b9 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.564407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:09.571948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30aa03-dc6b-4d2b-acc4-5d707ee39fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.572695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30aa03-dc6b-4d2b-acc4-5d707ee39fad None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.572980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30aa03-dc6b-4d2b-acc4-5d707ee39fad None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.573253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30aa03-dc6b-4d2b-acc4-5d707ee39fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.637121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:09.658582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce06ef-55e4-465a-ad7b-4485a0f60d51 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.658784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30aa03-dc6b-4d2b-acc4-5d707ee39fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:09.660224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30aa03-dc6b-4d2b-acc4-5d707ee39fad None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:09.699566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.700447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.700727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.701010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.710798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:09.712973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:09.731040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:09.732127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:09.733101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 318/637] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:38:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:09.734744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.777305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 320/638] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:09.778778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f65958-ed54-410e-8edc-9f733f5e4fff None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.779066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f65958-ed54-410e-8edc-9f733f5e4fff None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.779339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f65958-ed54-410e-8edc-9f733f5e4fff None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.780818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 321/639] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:09.781333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f65958-ed54-410e-8edc-9f733f5e4fff None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.807338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:09.855325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742f3fc-0e6e-4fe7-a017-64e2f15a7e5b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.891667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.892505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.892860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.893205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.893827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f65958-ed54-410e-8edc-9f733f5e4fff None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:09.900877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:09.904552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:09.912605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:09.913542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:09.914412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 319/640] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:09.915119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:09.936106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f2550-1275-4161-ae08-07fbb730dfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:09.936784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f2550-1275-4161-ae08-07fbb730dfff None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.937069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f2550-1275-4161-ae08-07fbb730dfff None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.937342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f2550-1275-4161-ae08-07fbb730dfff None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.938191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f2550-1275-4161-ae08-07fbb730dfff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:09.938502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f2550-1275-4161-ae08-07fbb730dfff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:09.939956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f2550-1275-4161-ae08-07fbb730dfff None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:09.957777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 322/641] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:38:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:09.959177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f25ba-d070-4653-a7a0-8a353997183c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:09.959529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f25ba-d070-4653-a7a0-8a353997183c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:09.959870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f25ba-d070-4653-a7a0-8a353997183c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:09.975653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f48f25ba-d070-4653-a7a0-8a353997183c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:09.976965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f48f25ba-d070-4653-a7a0-8a353997183c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:10.027060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655ce8a-4f64-4c70-967d-424f9bcb5cde None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.048685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 323/642] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:10.049838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f25ba-d070-4653-a7a0-8a353997183c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.062019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.078633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.079779 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.080068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.080343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.094842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.096717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:10.120602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:10.121625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:10.122554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:10.122554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:10.122554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:10.122554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:10.122554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 320/643] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:10.122846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.162149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f25ba-d070-4653-a7a0-8a353997183c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.177754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.200224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.200958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.201254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.201519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.202273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:10.202787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.204586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.213545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 324/644] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:10.214218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.234317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d89bbc-3016-4320-be90-51c39b97f078 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.274956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.275710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.276018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.276296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.291459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.327151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61d1c-bdff-4227-8ce5-d6c412730e3d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.359938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.361470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.380525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.381365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.381857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.382142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.396194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:10.410747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:10.411774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:10.412830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:10.418477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 325/645] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:10.418848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.445917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.518560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 321/646] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:10.518560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.552313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb3b8-8d99-4ca3-b829-007f041fbc4a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.588810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.596293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.599218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.599866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.599866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.600545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:10.602840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.605949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.631279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 326/647] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:10.631279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.639478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacddf9d-a6e7-4515-9fb2-492e343a29a4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.674534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.675345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.675717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.676099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.688980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.690558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:10.709304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:10.710687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:10.711601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:10.711601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:10.711601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:10.711601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:10.711601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 322/648] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:10.711601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.716286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.767409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c585ab-d869-4679-9764-3baad9703b6b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.802250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.803032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.803334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.803729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.804453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:10.804757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.806516 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.819775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 327/649] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:10.819775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.822533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f486cf-76e5-43a0-bc2a-b0b5c6e949bc None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.865566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.866377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.866666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.866943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.876812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.877728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.879281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:10.897099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:10.898242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:10.899261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:10.899261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:10.899261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:10.899261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:10.899261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 323/650] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:10.899652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:10.932536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da502da-71a2-488e-b367-4df59e956445 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:10.962770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:10.968736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:10.969547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:10.969827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:10.970090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:10.977172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:10.978635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:10.993710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:10.994640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 328/651] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:10.996328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.010382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f4fdf-e080-40d1-803b-015c76ab1373 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.046390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.047027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.047302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.047564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.048312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:11.048617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.049996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.061722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 324/652] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:11.062523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.093810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f057e1-2613-431a-aae3-a50b410d002a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.108048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.128317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75abe19b-c901-4702-b235-7443a16b7045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.129117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abe19b-c901-4702-b235-7443a16b7045 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.129394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abe19b-c901-4702-b235-7443a16b7045 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.129666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abe19b-c901-4702-b235-7443a16b7045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.139863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abe19b-c901-4702-b235-7443a16b7045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.141367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75abe19b-c901-4702-b235-7443a16b7045 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-75abe19b-c901-4702-b235-7443a16b7045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:11.158591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:11.159683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:11.161017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:11.161017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:11.161017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:11.161017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:11.161017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 329/653] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:11.161017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8751ad2-d61f-4067-99ae-b49c18d5da47 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.161428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8751ad2-d61f-4067-99ae-b49c18d5da47 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.161518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8751ad2-d61f-4067-99ae-b49c18d5da47 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.162770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 330/654] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:11.163456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8751ad2-d61f-4067-99ae-b49c18d5da47 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.165169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe372c-cf4d-4e01-a2ee-a48dda4609f2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.182334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.200626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.201247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.201529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.201804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.202524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:11.202831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.204231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.215300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 325/655] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:11.216103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.267387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8751ad2-d61f-4067-99ae-b49c18d5da47 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.304669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.305333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.305617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.305895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.306682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:11.307022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.308528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.317250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01679d82-4b26-482e-a815-320cbe02aa69 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.320033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 331/656] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:11.320700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.346634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.352448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993a359-b80c-4810-b139-c13bccac1fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.353250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993a359-b80c-4810-b139-c13bccac1fd3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.353532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993a359-b80c-4810-b139-c13bccac1fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.353808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993a359-b80c-4810-b139-c13bccac1fd3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.361383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993a359-b80c-4810-b139-c13bccac1fd3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.362867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993a359-b80c-4810-b139-c13bccac1fd3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5993a359-b80c-4810-b139-c13bccac1fd3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:11.376765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:11.377925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 326/657] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:11.379457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe62c34-5718-4ba4-bf46-9d447d8cd7d6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.380318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe62c34-5718-4ba4-bf46-9d447d8cd7d6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.380318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe62c34-5718-4ba4-bf46-9d447d8cd7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.396853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ebe62c34-5718-4ba4-bf46-9d447d8cd7d6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:11.398045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ebe62c34-5718-4ba4-bf46-9d447d8cd7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:11.422915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcdaca-1fce-432e-b7fe-77b085407472 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.426421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.457176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.457777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.458077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.458342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.459061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:11.459367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.460786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.464649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 327/658] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:11.465475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe62c34-5718-4ba4-bf46-9d447d8cd7d6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.481566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 332/659] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:11.482637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.563923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe62c34-5718-4ba4-bf46-9d447d8cd7d6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.567652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.582454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e955b4-8fff-47a7-867d-00132149da74 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.599608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.600417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.600698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.600966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.607856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.609439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.615961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.616651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.616930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.617209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.618034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=7c9cdda3d7884298a791f90523f3ab91) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:11.618344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.619754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:11.622944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:11.623929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:11.624873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 328/660] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:11.625372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.625823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 333/661] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/role_assignments?user.id=7c9cdda3d7884298a791f90523f3ab91&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:11.626312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.663450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.742197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb4032-8761-42b7-9995-2bd75f835410 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.745608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa1b94-a45b-44a2-a138-ef212288c19b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.781448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.782265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.782553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.782825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.786050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.786731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.787011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.787284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.788167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:11.788473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.789926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.791143 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.794740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.808980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 329/662] 10.222.0.50 () {56 vars in 1137 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:11.809381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:11.810526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 15:38:11.811716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:11.812453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.812536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 334/663] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:11.812536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.827064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.903625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:11.920832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bbd56-a659-4d52-8dd7-b200ce19a6c5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.932789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4c466-1fe8-4790-9b61-b5fc219913f0 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:11.956514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.957321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.957603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.957877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.967575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:11.968191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:11.968425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:11.968530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:11.968724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:11.969511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:11.969814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:11.971264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:11.976081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:11.977031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:11.977869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 330/664] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:11.978570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:11.988113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 335/665] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:11.988113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.060957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.089474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd29a77-0f0e-4aa8-bdcb-b62e3f42b6dd None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.093702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eff927-4ed0-46cc-acfe-f7ebac649a33 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.125636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.126285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.126564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.126834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.130234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.130918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.131200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.131474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.132360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:12.132673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.134599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.143121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.156537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 336/666] 10.222.0.50 () {56 vars in 1139 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:12.156537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.203867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.205450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.252344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f55db-6366-4316-b8d8-21a9540c97ca None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.282054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.290801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.291478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.292186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.292186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.292871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:12.293173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.294548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.303432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 337/667] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:12.308257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.326224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 331/668] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:12.326224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.367483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.423929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7a643-adfd-4ec1-80f1-0ff7c78bcde8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.425413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c74c1-db39-403f-8ef1-818bd4ef0763 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.459971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.460488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.460634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.460915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.461189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.461280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.461620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.461898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.463252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=4da1612f1c1740ceb2a918f0a8dd057b) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:12.463565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.465529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.480020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 338/669] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/role_assignments?user.id=4da1612f1c1740ceb2a918f0a8dd057b&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:12.480020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.549932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.551703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.559070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.605589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.622653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa72a8-ac65-48f0-b514-d695da32d0cc None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.669720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.669720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.670279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.670556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.681321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 332/670] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:12.681321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.682138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.684104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:12.702634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:12.703584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:12.704505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:12.704505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:12.704505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:12.704505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:12.704505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 339/671] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:12.705225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.808065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a682b0-41b0-42eb-a51f-c3f006b0e292 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.830825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5257d47-744f-420a-918d-f26a8e1f4c88 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:12.838087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.846454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.847263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.847544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.847817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.859379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.860894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.863621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:12.866701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:12.867506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:12.868370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:12.868370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:12.879256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:12.880454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:12.881400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:12.882285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:12.882285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:12.882285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:12.882285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:12.882892 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:12.883105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 333/672] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:12.883105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:12.904966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:12.906854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:12.909048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:12.909048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:12.909048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:12.909048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:12.909048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 340/673] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:12.909048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:12.989997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b925cf-3518-4d26-8da2-0134fb1df200 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.008534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e9319-5e15-46b3-9eb1-e35c585ef3c4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.026869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.027534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.027816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.028133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.029001 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:13.029315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.030711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.042984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.043651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.043927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.044233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.044948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:13.045250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.046609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.048110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 334/674] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:13.049163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.063513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 341/675] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:13.064434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.083112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.113450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.163114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e3ade-fb3a-44ec-9632-20dff545ce5d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.165115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29670c4-1cbf-4f3a-904f-f7f73a0903af None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.199778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.200840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.201222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.201462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.201590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.202408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.202683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.202952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.203737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:13.204184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.205651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.210030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.211623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.218073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 335/676] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:13.219349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:13.227585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:13.228694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:13.229820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:13.232699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 342/677] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:13.233796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.316127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.338468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e954b70-b77b-45d1-9200-29500546bad6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.343854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f259596-f18f-417c-87f9-17df2c2292a9 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.361390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.374724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.375858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.376361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.376840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.381780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.382582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.382862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.383133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.385830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.387477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.390665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.392117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:13.402100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:13.402981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:13.403941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:13.406455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 343/678] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:13.407446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:13.408101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:13.408948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:13.409844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:13.413852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 336/679] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:13.415418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.512343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5470ea-d107-4116-8513-699003a6cbd1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.524219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f7de5-d503-4eeb-af6d-7567c8c934aa None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.547695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.548719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.549242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.549731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.550967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:13.551497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.553172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.560910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.561578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.561859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.562130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.562914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:13.563221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.563888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.566064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.574135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 344/680] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:13.575198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.584013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 337/681] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:13.585086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.623206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.675962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ad71-7a04-4816-b38f-0a46cd7a175a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.696308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008c3e5-2c7d-47ab-9acb-293ff057bde1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.710905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.711767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.712179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.712810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.713445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:13.713885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.715485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.729208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 345/682] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:13.735197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.737278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02feb059-bde7-4348-812e-96d86060a26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.738220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feb059-bde7-4348-812e-96d86060a26e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.738674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feb059-bde7-4348-812e-96d86060a26e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.739119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02feb059-bde7-4348-812e-96d86060a26e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.792261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.813197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02feb059-bde7-4348-812e-96d86060a26e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.814661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02feb059-bde7-4348-812e-96d86060a26e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.839501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae10e1-7a94-4704-af45-66f9c15395ea None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:13.850754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:13.873664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:13.874624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.874904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.875183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.876030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=5e17d25a83fe48d99ddbd209fbea8486) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:13.876350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:13.877789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:13.884502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 346/683] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/role_assignments?user.id=5e17d25a83fe48d99ddbd209fbea8486&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:13.889577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:13.946051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 338/684] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:13.946051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edff42-97e4-44e5-adf8-324daa2de6b5 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:13.946430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edff42-97e4-44e5-adf8-324daa2de6b5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:13.946430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82edff42-97e4-44e5-adf8-324daa2de6b5 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:13.948144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 339/685] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:13.949040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edff42-97e4-44e5-adf8-324daa2de6b5 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.032748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0cd8e2-55c9-42f6-9685-c9fdfdde03ad None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.057296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:14.076740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c5e0a-89a4-42ae-bf1e-a3fa589d7fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.077405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82edff42-97e4-44e5-adf8-324daa2de6b5 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.077786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c5e0a-89a4-42ae-bf1e-a3fa589d7fe2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.078158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c5e0a-89a4-42ae-bf1e-a3fa589d7fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.078432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c5e0a-89a4-42ae-bf1e-a3fa589d7fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.079277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c5e0a-89a4-42ae-bf1e-a3fa589d7fe2 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=e4cb4d5213c647459e1c21c311a4a305) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:14.079631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c5e0a-89a4-42ae-bf1e-a3fa589d7fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.081181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c5e0a-89a4-42ae-bf1e-a3fa589d7fe2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.088452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 347/686] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/role_assignments?user.id=e4cb4d5213c647459e1c21c311a4a305&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:14.089098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3cbd6-cbd0-4e3f-8b36-cbd69d3daedc None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.089391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3cbd6-cbd0-4e3f-8b36-cbd69d3daedc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.089667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3cbd6-cbd0-4e3f-8b36-cbd69d3daedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.108953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-86d3cbd6-cbd0-4e3f-8b36-cbd69d3daedc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:14.110795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-86d3cbd6-cbd0-4e3f-8b36-cbd69d3daedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:14.119648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.121724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.122219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.122672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.128430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:14.140436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.142461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:14.166988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:14.168072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:14.169045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:14.169045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:14.169045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:14.169045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:14.172419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 340/687] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:14.179667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.207262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 348/688] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:14.240127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3cbd6-cbd0-4e3f-8b36-cbd69d3daedc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.312085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d72d0e-5d24-467e-901a-935b19852331 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.347668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.349033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.349496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.349934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.351148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:14.351653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.353457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.371256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 341/689] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:14.372567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.372981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3cbd6-cbd0-4e3f-8b36-cbd69d3daedc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.388970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:14.411101 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.412117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.412491 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.412858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.414420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.415204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:14.434049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:14.435329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:14.436434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 349/690] 10.222.0.50 () {56 vars in 1129 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:14.444040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.446477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:14.509726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1a4ba-a281-4f2a-9ffe-92983f710e46 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.555478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.556580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.556865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.557383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.563878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df151107-079a-4d55-ab32-7dfaeb2be536 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.566580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.569047 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:14.588127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:14.589233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:14.590737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 342/691] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:14.597490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.607403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.608122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.608461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.608733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.609497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:14.609804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.611239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.616495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 350/692] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:14.621774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.678702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:14.723094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a82a0-e1dd-442f-bc65-0bd3eaec68ee None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.732821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:14.740286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6e45e-3eaa-46ad-b38e-a87f90872c5e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.760354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c56b0-5b97-48e5-8227-6183a9a20014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.761154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c56b0-5b97-48e5-8227-6183a9a20014 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.761553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c56b0-5b97-48e5-8227-6183a9a20014 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.761919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c56b0-5b97-48e5-8227-6183a9a20014 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.762792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c56b0-5b97-48e5-8227-6183a9a20014 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:14.763232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c56b0-5b97-48e5-8227-6183a9a20014 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.764935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3c56b0-5b97-48e5-8227-6183a9a20014 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.778300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.782535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 343/693] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:14.782535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae49d8d-90ae-4935-adac-c5d12c35e4e5 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.782535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae49d8d-90ae-4935-adac-c5d12c35e4e5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.782535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae49d8d-90ae-4935-adac-c5d12c35e4e5 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.782535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 344/694] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:14.783226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.783589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.783873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.784634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:14.784950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.788029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:14.788976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a88cd-6c5b-4b61-8d91-bd38218902e9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.789450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a88cd-6c5b-4b61-8d91-bd38218902e9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.789908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a88cd-6c5b-4b61-8d91-bd38218902e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.795091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 351/695] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:14.795973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.809209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-651a88cd-6c5b-4b61-8d91-bd38218902e9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:14.810967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-651a88cd-6c5b-4b61-8d91-bd38218902e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:14.886264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 345/696] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:38:14.914198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a88cd-6c5b-4b61-8d91-bd38218902e9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:14.919042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853be01-83c5-45e3-9d80-6c398ac6438e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:14.944423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:14.986383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:14.987439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:14.987963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:14.988481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:14.989530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2933f88027fc435eba3064124e2dd5d5) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:14.990053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:14.992924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:15.005093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 352/697] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/role_assignments?user.id=2933f88027fc435eba3064124e2dd5d5&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:15.058908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:15.063883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a88cd-6c5b-4b61-8d91-bd38218902e9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:15.104205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:15.105080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:15.105362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:15.105639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:15.117522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:15.125639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:15.126652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:15.127531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 346/698] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:15.134202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:15.243342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c3141-0fef-492a-8865-9ae7f7ac7773 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:15.295271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:15.295959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:15.296277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:15.296549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:15.297318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:15.297624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:15.299010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:15.309717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 353/699] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:15.315765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:15.419053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ba00-331c-4062-b8c2-043848c92722 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:15.453372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:15.454219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:15.454543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:15.454821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:15.465964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:15.467510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:15.484956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:15.485950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:15.486873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:15.486873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:15.486873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:15.486873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:15.486873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 347/700] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:15.492039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:15.525783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:15.601270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9fe37-aa60-4cd4-85fb-40e1838c9f20 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:15.638241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:15.638944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:15.639221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:15.639496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:15.640248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:15.640558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:15.641941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:15.654199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 354/701] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:15.659407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:15.681328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:15.818258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24391f10-46a6-4532-94c7-b139e20f364c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:15.873701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:15.874430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:15.874713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:15.874989 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:15.889134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:15.890602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:15.908220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:15.909293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:15.910929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 348/702] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:38:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:15.915046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:16.095131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbad2dc-49a9-4811-a4e3-f3c6bf4a3936 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:16.096728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:16.147411 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba9f01-96cf-43ee-b8c8-e2b03a799315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:16.148070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba9f01-96cf-43ee-b8c8-e2b03a799315 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:16.148315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba9f01-96cf-43ee-b8c8-e2b03a799315 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:16.148548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba9f01-96cf-43ee-b8c8-e2b03a799315 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:16.149384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba9f01-96cf-43ee-b8c8-e2b03a799315 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:16.149688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba9f01-96cf-43ee-b8c8-e2b03a799315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:16.151119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba9f01-96cf-43ee-b8c8-e2b03a799315 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:16.177119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 355/703] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:38:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:16.182293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:16.308514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:16.349976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fec7f-54a2-4d7b-b041-a5cc974d7400 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:16.393541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:16.394195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:16.394478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:16.394750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:16.485476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:16.487055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:16.488429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16215838-394e-4d7c-9ba2-5c0e9365d7b4 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:16.488717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16215838-394e-4d7c-9ba2-5c0e9365d7b4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:16.488989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16215838-394e-4d7c-9ba2-5c0e9365d7b4 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:16.490572 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 356/704] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:16.496723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4775d-b501-4197-b5bb-33b97275b4c8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:16.497011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4775d-b501-4197-b5bb-33b97275b4c8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:16.497293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4775d-b501-4197-b5bb-33b97275b4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:16.527216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-35d4775d-b501-4197-b5bb-33b97275b4c8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:16.528447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-35d4775d-b501-4197-b5bb-33b97275b4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:16.616344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:16.664030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 357/705] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:16.665598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 349/706] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:16.673105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4775d-b501-4197-b5bb-33b97275b4c8 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:16.695605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:16.803579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcd4c8-8cb5-4e31-a147-14086ed97317 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:16.821472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:16.856794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4775d-b501-4197-b5bb-33b97275b4c8 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:16.864014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:16.864014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:16.864014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:16.864014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] PATH_INFO: `/v3/services/network` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:16.864391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:16.864391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:16.877061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:16.878127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:16.878971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:16.883447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 350/707] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:16 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:16.885008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:16.922082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:16.937592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:16.938799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:16.944376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:16.944663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:16.962009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:16.963465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:17.005151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:17.006106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:17.007043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:17.007043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:17.007043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:17.007043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:17.007043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 358/708] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:17.007327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6cd0-df82-4d3d-bac3-3e18ac8a7847 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.010035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.070576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.071166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.071449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.071724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.072545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] RBAC: Adding query filter params (name=network) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:17.072849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.074268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.093834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 351/709] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:17.100940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.128463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d821a45-ec83-468d-8684-0cb7d286cc15 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.160543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:17.164686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5191c-9468-43cb-8144-e294af697e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.165303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5191c-9468-43cb-8144-e294af697e88 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.165587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5191c-9468-43cb-8144-e294af697e88 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.165864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5191c-9468-43cb-8144-e294af697e88 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.166630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5191c-9468-43cb-8144-e294af697e88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:17.166966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5191c-9468-43cb-8144-e294af697e88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.168492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5191c-9468-43cb-8144-e294af697e88 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.183752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 359/710] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:17.197641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af57f51-1e3e-4289-ba63-c7cedf93a2fa None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.197956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af57f51-1e3e-4289-ba63-c7cedf93a2fa None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.198227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af57f51-1e3e-4289-ba63-c7cedf93a2fa None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.199713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 360/711] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:17.200290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af57f51-1e3e-4289-ba63-c7cedf93a2fa None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.262156 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:17.268524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31949947-1e90-4c6c-98ad-7494755fc4f3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.315674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6162d-0b7c-40dd-8b0c-786727c4f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.316603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6162d-0b7c-40dd-8b0c-786727c4f668 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.317020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6162d-0b7c-40dd-8b0c-786727c4f668 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.317307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6162d-0b7c-40dd-8b0c-786727c4f668 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.318068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6162d-0b7c-40dd-8b0c-786727c4f668 None admin] RBAC: Adding query filter params (type=network) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:17.318381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6162d-0b7c-40dd-8b0c-786727c4f668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.320538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6162d-0b7c-40dd-8b0c-786727c4f668 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.336120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 352/712] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:17.336120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8cdb0-61ef-45c9-ba19-a0abf3935f46 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.336120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8cdb0-61ef-45c9-ba19-a0abf3935f46 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.336120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8cdb0-61ef-45c9-ba19-a0abf3935f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.350718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f0f8cdb0-61ef-45c9-ba19-a0abf3935f46 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:17.352204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f0f8cdb0-61ef-45c9-ba19-a0abf3935f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:17.352989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af57f51-1e3e-4289-ba63-c7cedf93a2fa None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.391973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023baa7-9f40-4705-9d1a-b914c92d686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.392828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023baa7-9f40-4705-9d1a-b914c92d686d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.393124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023baa7-9f40-4705-9d1a-b914c92d686d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.393384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023baa7-9f40-4705-9d1a-b914c92d686d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.401465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023baa7-9f40-4705-9d1a-b914c92d686d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.402980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023baa7-9f40-4705-9d1a-b914c92d686d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-1023baa7-9f40-4705-9d1a-b914c92d686d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:17.418656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:17.419743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:17.420672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:17.424391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 361/713] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:17.425877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8db36-1a10-46c0-b32a-11af33e47db0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.426168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8db36-1a10-46c0-b32a-11af33e47db0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.426446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8db36-1a10-46c0-b32a-11af33e47db0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.432227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 362/714] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:17.433198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8db36-1a10-46c0-b32a-11af33e47db0 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.439825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 353/715] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:17.441233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cd198-95df-40b3-91f2-3c9f4a2d5d3d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.441523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cd198-95df-40b3-91f2-3c9f4a2d5d3d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.441805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cd198-95df-40b3-91f2-3c9f4a2d5d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.464223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1f2cd198-95df-40b3-91f2-3c9f4a2d5d3d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:17.465720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1f2cd198-95df-40b3-91f2-3c9f4a2d5d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:17.497268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:17.546319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 354/716] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 15:38:17.547204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cd198-95df-40b3-91f2-3c9f4a2d5d3d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.559192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8db36-1a10-46c0-b32a-11af33e47db0 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.579532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:17.598354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.598950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.599231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.599503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.600326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:17.600637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.602085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.616386 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 363/717] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:17.617444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.663348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cd198-95df-40b3-91f2-3c9f4a2d5d3d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.700082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.701022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.701309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.701587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.712428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.713974 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:17.732295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:17.733275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:17.734178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:17.734178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:17.734178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:17.734178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:17.734178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 355/718] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:17.734770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.754953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5340f-aee3-4c46-8897-12f6a21316de None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.772164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:17.794816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.795487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.795769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.796075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.796807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.798193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.814469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 364/719] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:17 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:17.815364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:17.838102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:17.871573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de6e6e-8b80-4b43-8df2-1b0fb3cc03df None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.919366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa307a-0ff8-405c-8c9a-df1e6727f8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:17.920397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa307a-0ff8-405c-8c9a-df1e6727f8ca None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.920916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa307a-0ff8-405c-8c9a-df1e6727f8ca None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.921407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa307a-0ff8-405c-8c9a-df1e6727f8ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.922485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa307a-0ff8-405c-8c9a-df1e6727f8ca None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=4da1612f1c1740ceb2a918f0a8dd057b) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:17.923061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa307a-0ff8-405c-8c9a-df1e6727f8ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:17.924309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa307a-0ff8-405c-8c9a-df1e6727f8ca None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:17.936159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 356/720] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:17 2022] GET /identity/v3/role_assignments?user.id=4da1612f1c1740ceb2a918f0a8dd057b&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:17.936159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483fb1-259c-4157-8536-c4b4f8a2f469 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.936159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483fb1-259c-4157-8536-c4b4f8a2f469 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.936159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2483fb1-259c-4157-8536-c4b4f8a2f469 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.936159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 357/721] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:17.940511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728e983-1c61-4524-8fbd-35585c472b04 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:17.940837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728e983-1c61-4524-8fbd-35585c472b04 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:17.941066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728e983-1c61-4524-8fbd-35585c472b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:17.946103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954bc35-2b80-4257-9c41-0fbda49d083a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:17.973806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7728e983-1c61-4524-8fbd-35585c472b04 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:17.975177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7728e983-1c61-4524-8fbd-35585c472b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:18.016256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:18.017041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:18.017326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:18.017923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:18.018678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:18.018987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:18.020464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:18.040105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 365/722] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:18.045784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:18.080329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 358/723] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:38:18.092683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:18.104210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728e983-1c61-4524-8fbd-35585c472b04 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:18.188165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2893e9b-4781-4146-ae75-69c840f8543b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:18.192066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:18.257692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728e983-1c61-4524-8fbd-35585c472b04 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:18.269325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:18.270035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:18.270319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:18.270599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:18.271326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:18.272809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:18.299475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:18.300383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:18.300678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:18.300935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:18.317766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 366/724] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:38:18 2022] POST /identity/v3/users => generated 254 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:18.320929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:18.329444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:18.330702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:18.331807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:18.334294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 359/725] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:18.340480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:18.463489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:18.483504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd992c9-1d58-4127-98b9-a1ee554d1fae None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:18.534645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:18.535632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:18.536075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:18.536483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:18.537478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:18.537925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:18.539472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:18.555564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 367/726] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:18.561365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:18.675674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ce261-2634-462b-b75f-06f62c377a4e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:18.704822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:18.719159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:18.720234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:18.720509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:18.720774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:18.732128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:18.733603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:18.754552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:18.755586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:18.756616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:18.756616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:18.756616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:18.756616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:18.756616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 360/727] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:18.763510 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:18.878752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bdaf0-3801-4e52-98f0-600f299bbefe None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:18.914365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:18.915413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:18.915890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:18.916381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:18.917365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:18.917895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:18.919464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:18.936203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 368/728] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:18.937893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:19.045864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daee1b9-88c0-4e1f-9f45-55f3eb31610f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:19.083706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:19.084919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:19.085400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:19.085865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:19.093682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:19.095351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:19.111910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:19.113011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:19.113965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:19.116651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 361/729] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:19.122540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:19.136204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:19.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f0824-4146-4304-873a-5dd29f662137 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:19.271930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:19.272647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:19.272925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:19.273198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:19.274039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:19.274348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:19.276529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:19.294850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 369/730] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Jun 30 15:38:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:19.300712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:19.337617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:19.422414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6336e99-2831-4150-8c2f-98a2b48696fa None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:19.456902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:19.457551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:19.457828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:19.458094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:19.538635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:19.540498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:19.592018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:19.666638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 362/731] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 15:38:19.671912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:19.777227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:19.822812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e35fb6-fc08-42f4-b1ba-4f893245947a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:19.867104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cad7a-9ca1-4f1e-8661-a7993ff104f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:19.868241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cad7a-9ca1-4f1e-8661-a7993ff104f2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:19.868702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cad7a-9ca1-4f1e-8661-a7993ff104f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:19.869171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cad7a-9ca1-4f1e-8661-a7993ff104f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:19.890124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cad7a-9ca1-4f1e-8661-a7993ff104f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:19.891965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cad7a-9ca1-4f1e-8661-a7993ff104f2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-5e5cad7a-9ca1-4f1e-8661-a7993ff104f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:19.915693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:19.916775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:19.917683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:19.917683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:19.917683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:19.917683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:19.919335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 370/732] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:19.925224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:19.935188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b0cf-a7ba-44a7-9409-a8c0d74775f3 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:19.935467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b0cf-a7ba-44a7-9409-a8c0d74775f3 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:19.935742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4b0cf-a7ba-44a7-9409-a8c0d74775f3 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:19.937389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 371/733] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:19.942178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753107d6-2f18-427c-9572-19abf3b7acd5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:19.942461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753107d6-2f18-427c-9572-19abf3b7acd5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:19.942738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753107d6-2f18-427c-9572-19abf3b7acd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:19.959642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-753107d6-2f18-427c-9572-19abf3b7acd5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:19.960841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-753107d6-2f18-427c-9572-19abf3b7acd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:20.030647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 372/734] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 15:38:20.053682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753107d6-2f18-427c-9572-19abf3b7acd5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:20.104908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:20.134771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f31af4-44e5-4b78-b9a8-489138591410 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:20.162251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753107d6-2f18-427c-9572-19abf3b7acd5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:20.199437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:20.200141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.200420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.200811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.201901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:20.203303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:20.225778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 373/735] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:20 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:20.233214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:20.238069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1c339-2a72-4ba3-8ae8-09ea94803d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:20.238817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c339-2a72-4ba3-8ae8-09ea94803d1d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.239092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c339-2a72-4ba3-8ae8-09ea94803d1d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.239358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1c339-2a72-4ba3-8ae8-09ea94803d1d None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.240182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1c339-2a72-4ba3-8ae8-09ea94803d1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:20.240639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1c339-2a72-4ba3-8ae8-09ea94803d1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:20.242146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1c339-2a72-4ba3-8ae8-09ea94803d1d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:20.260726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 363/736] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:20.268665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:20.324175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52dd98-a19b-457f-be81-dc4da1fe44dc None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.324524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52dd98-a19b-457f-be81-dc4da1fe44dc None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.324717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52dd98-a19b-457f-be81-dc4da1fe44dc None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.326364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 364/737] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:20.336400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d7b78-7c90-4d4f-b7fe-7b9b9e093a73 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.336740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d7b78-7c90-4d4f-b7fe-7b9b9e093a73 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.336954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d7b78-7c90-4d4f-b7fe-7b9b9e093a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.368370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-2a8d7b78-7c90-4d4f-b7fe-7b9b9e093a73 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:20.369611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2a8d7b78-7c90-4d4f-b7fe-7b9b9e093a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:20.476318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d3717-d331-4391-a7da-bd7284c65a44 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:20.491267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 365/738] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:20.492429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:20.516845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d7b78-7c90-4d4f-b7fe-7b9b9e093a73 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:20.527970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:20.529047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.529479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.529900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.539364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:20.541625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:20.559113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:20.560289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:20.561441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 374/739] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:20.567171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:20.653909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d7b78-7c90-4d4f-b7fe-7b9b9e093a73 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:20.685288 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:20.698714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e18d4e-522a-470d-8ca1-59cf1309439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:20.699520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb81c2a-ae9b-4edc-86ec-feec8e0585fe None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:20.699712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18d4e-522a-470d-8ca1-59cf1309439c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.700016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18d4e-522a-470d-8ca1-59cf1309439c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.700300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e18d4e-522a-470d-8ca1-59cf1309439c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.711437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e18d4e-522a-470d-8ca1-59cf1309439c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-21e18d4e-522a-470d-8ca1-59cf1309439c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:20.719301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:20.720286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:20.721149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 366/740] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:20.722340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2f554-f008-4725-8a9e-a9ea913e7889 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.722449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2f554-f008-4725-8a9e-a9ea913e7889 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.722640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2f554-f008-4725-8a9e-a9ea913e7889 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.728971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 367/741] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:20.729846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2f554-f008-4725-8a9e-a9ea913e7889 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:20.748702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf691e7-5845-46aa-b8ea-77e92afc6a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:20.749372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf691e7-5845-46aa-b8ea-77e92afc6a20 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.749647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf691e7-5845-46aa-b8ea-77e92afc6a20 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.749915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf691e7-5845-46aa-b8ea-77e92afc6a20 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.750708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf691e7-5845-46aa-b8ea-77e92afc6a20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:20.751014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf691e7-5845-46aa-b8ea-77e92afc6a20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:20.752487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf691e7-5845-46aa-b8ea-77e92afc6a20 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:20.767155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 375/742] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:20.768544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1cd7-992d-4bc6-863c-1751cf122ceb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.768887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1cd7-992d-4bc6-863c-1751cf122ceb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.769088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1cd7-992d-4bc6-863c-1751cf122ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.786482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e1be1cd7-992d-4bc6-863c-1751cf122ceb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:20.788020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e1be1cd7-992d-4bc6-863c-1751cf122ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:20.813716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:20.865262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2f554-f008-4725-8a9e-a9ea913e7889 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:20.876899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 376/743] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:20.878131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be1cd7-992d-4bc6-863c-1751cf122ceb None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:20.917948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:20.918618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:20.918895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:20.919163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:20.919908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:20.920532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:20.921914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:20.932022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 368/744] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:20.933055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:20.993375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.010881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be1cd7-992d-4bc6-863c-1751cf122ceb None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.047550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.048684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.049191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.049678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.050790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2933f88027fc435eba3064124e2dd5d5) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:21.051307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.053070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.059619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 377/745] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:20 2022] GET /identity/v3/role_assignments?user.id=2933f88027fc435eba3064124e2dd5d5&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:21.060718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.076312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd9d78-09cf-452c-8cfc-cdc1820709f1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.109443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.125197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6aa53-fb63-4e4d-9644-2cf9ce97f739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.126082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6aa53-fb63-4e4d-9644-2cf9ce97f739 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.126364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6aa53-fb63-4e4d-9644-2cf9ce97f739 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.126638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6aa53-fb63-4e4d-9644-2cf9ce97f739 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.127305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6aa53-fb63-4e4d-9644-2cf9ce97f739 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.128819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6aa53-fb63-4e4d-9644-2cf9ce97f739 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-9da6aa53-fb63-4e4d-9644-2cf9ce97f739 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:21.142840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:21.144358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:21.145193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 369/746] 10.222.0.50 () {56 vars in 1129 bytes} [Thu Jun 30 15:38:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:21.146048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a022-5fed-4f5d-92bc-e74db7905460 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.146668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a022-5fed-4f5d-92bc-e74db7905460 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.146668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120a022-5fed-4f5d-92bc-e74db7905460 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.151975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 370/747] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:21.152925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120a022-5fed-4f5d-92bc-e74db7905460 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.198423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b8dc1-0061-4bbc-86ef-ee0b66b7f131 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.235833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e0ecc-c8f0-4770-a21c-208425ddbb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.236698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e0ecc-c8f0-4770-a21c-208425ddbb2d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.236972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e0ecc-c8f0-4770-a21c-208425ddbb2d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.237241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e0ecc-c8f0-4770-a21c-208425ddbb2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.248797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e0ecc-c8f0-4770-a21c-208425ddbb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.250293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e0ecc-c8f0-4770-a21c-208425ddbb2d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.269534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120a022-5fed-4f5d-92bc-e74db7905460 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-140e0ecc-c8f0-4770-a21c-208425ddbb2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:21.269659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:21.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:21.271802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:21.271802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:21.271802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:21.271802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:21.271802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 378/748] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:21.272708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773ad6-3eef-49a9-8555-d24981467411 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.272791 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773ad6-3eef-49a9-8555-d24981467411 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.276145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c773ad6-3eef-49a9-8555-d24981467411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.284394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.303257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3c773ad6-3eef-49a9-8555-d24981467411 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:21.305164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3c773ad6-3eef-49a9-8555-d24981467411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:21.310028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.310709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.310995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.311275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.312018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:21.312323 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.313811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.319103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 371/749] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:21.319971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.394612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 379/750] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:21.395301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c773ad6-3eef-49a9-8555-d24981467411 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.404232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.472196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba42e9d-7f85-4632-a8c3-30aa1145c1fc None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.508114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.509122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.509405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.509684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.510445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:21.510753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.512252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.516753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c773ad6-3eef-49a9-8555-d24981467411 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.530914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 372/751] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:21.532050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.555905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.557168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.557741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.558216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.559135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:21.559655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.561413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.567519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 380/752] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:21.568698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.569925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.639908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.643572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800a5e2-6925-4b15-940f-61311b2c7657 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.671903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c733c-3457-4c5e-9785-b7a73f7c20fb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.680230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.681045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.681687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.681687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.692873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:21.700807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:21.701757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:21.702626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:21.707005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 373/753] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:21.707867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.713731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.714874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.715374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.715844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.723964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.725668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:21.740835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:21.741815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:21.742772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:21.749921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 381/754] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:21.751040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.805209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.813514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f227eee-69bf-403e-b3b5-def2b7390df7 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.850712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.851480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.851769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.852085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.852930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] RBAC: Adding query filter params (interface=public, service_id=be42f16847a545d49ddbb94de5b0398a, region_id=RegionOne) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:21.853197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.854675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.855396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c06b7-718d-44fd-8104-008e17ab9499 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:21.863391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 374/755] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/endpoints?service_id=be42f16847a545d49ddbb94de5b0398a&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:21.864256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.886127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:21.892709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:21.893654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:21.894134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:21.894605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:21.895582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:21.896125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:21.897710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:21.906611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 382/756] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:21.907708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:21.976686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020683b-5854-4172-b8b2-19c01bd5375d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.006086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a74c79-3ca8-4e9f-abe4-2f1457b8f1cb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.012952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.014184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.014567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.015023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.016140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:22.016572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.018511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.043432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 375/757] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:22.043432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.048688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.049592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.050048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.050511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.051383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.052991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.058895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 383/758] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:21 2022] GET /identity/v3/services => generated 772 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:22.059951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.064401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:22.120577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:22.168186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b1774-a290-4a68-9d97-354bee0729d4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.205810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.207059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.207565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.208091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.220481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.222306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.236485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2e097-0e07-4ed3-864a-364b977f96b9 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:22.241692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:22.242659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:22.243554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:22.243554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:22.243554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:22.243554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:22.245930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 384/759] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:22.252011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.396905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c0df6-53cf-4235-95cf-70545ba07a88 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.409153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:22.434258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.434593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.434850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.435123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.438232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.439151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.439631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.440212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.441792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:22.442336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.443929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.446203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.447753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.457547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 385/760] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:22.463315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:22.466979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:22.467866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:22.469635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:22.469635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:22.469635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:22.469635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:22.472231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 376/761] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:22.474212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.556393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:22.578888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02ad9dc-684a-49a4-9976-5fe1617a7416 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.591216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72562ca-712e-40cb-9d76-e3df093aee71 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.613991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.615184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.615693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.616193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.623468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.625485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.627218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.628010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.628302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.628575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.629444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:22.629751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.631128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:22.641867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:22.642771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:22.643657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:22.646073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 386/763] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:22.646178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 377/763] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:22.650882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.652035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.690478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:22.765646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cd26e-e284-48f1-abde-00071704c000 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.772150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1984-dde4-4e0d-99e1-2ed649d89f67 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.800143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:22.802230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627fb2b-96eb-4a20-ab7d-8207dbff25f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.803112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627fb2b-96eb-4a20-ab7d-8207dbff25f2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.803552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627fb2b-96eb-4a20-ab7d-8207dbff25f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.803973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627fb2b-96eb-4a20-ab7d-8207dbff25f2 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.804972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627fb2b-96eb-4a20-ab7d-8207dbff25f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:22.805431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627fb2b-96eb-4a20-ab7d-8207dbff25f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.807341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627fb2b-96eb-4a20-ab7d-8207dbff25f2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.809800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:22.810974 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.811592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.812064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.820136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:22.821710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:22.823068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 387/764] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:22.824848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade125-4434-491f-902f-2a0d4c1ca131 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.825371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade125-4434-491f-902f-2a0d4c1ca131 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.825860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade125-4434-491f-902f-2a0d4c1ca131 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.831663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 388/765] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:22.833171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407e97-4781-49f3-a44a-bcf740ee6b10 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:22.833660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407e97-4781-49f3-a44a-bcf740ee6b10 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:22.834156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407e97-4781-49f3-a44a-bcf740ee6b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:22.838082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:22.839052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:22.840028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:22.845080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 378/766] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:22.845991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.856261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9f407e97-4781-49f3-a44a-bcf740ee6b10 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:22.857963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9f407e97-4781-49f3-a44a-bcf740ee6b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:22.938194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 389/767] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:22.938982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f407e97-4781-49f3-a44a-bcf740ee6b10 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:22.953419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fe13a-1674-476f-8d96-f5e36314d0be None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:22.957946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:22.998323 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.000486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.000763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.001047 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/4da1612f1c1740ceb2a918f0a8dd057b/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.004830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=4da1612f1c1740ceb2a918f0a8dd057b, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.007152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.028533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 379/768] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:22 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/4da1612f1c1740ceb2a918f0a8dd057b/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:23.029841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.060479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f407e97-4781-49f3-a44a-bcf740ee6b10 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.103399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:23.105067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.106683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.106683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.106683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.107390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:23.107854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.109371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.135580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 390/769] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Jun 30 15:38:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:23.141472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.183510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fd677-cc5a-4186-b7b0-8881d47ee185 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.236934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.237962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.238249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.238624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] PATH_INFO: `/v3/services/network` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.239306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.240747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:23.259115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:23.260186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:23.261073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 380/770] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:23.275145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.283920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:23.310555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cb96c-b8bc-47d1-876b-a8924176ace7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.347597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfd2a8-43c0-4dd2-bdd4-08cef06e3ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.348289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd2a8-43c0-4dd2-bdd4-08cef06e3ddd None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.348572 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd2a8-43c0-4dd2-bdd4-08cef06e3ddd None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.348844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfd2a8-43c0-4dd2-bdd4-08cef06e3ddd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.413271 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88605b97-f0fc-4baf-9402-864826f2ff89 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.426671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:23.428888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfd2a8-43c0-4dd2-bdd4-08cef06e3ddd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.431504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfd2a8-43c0-4dd2-bdd4-08cef06e3ddd None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.453925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f45c3-24b7-40db-b818-b4f905f3b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.454821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f45c3-24b7-40db-b818-b4f905f3b931 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.455284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f45c3-24b7-40db-b818-b4f905f3b931 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.455503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f45c3-24b7-40db-b818-b4f905f3b931 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.457026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f45c3-24b7-40db-b818-b4f905f3b931 None admin] RBAC: Adding query filter params (name=network) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:23.457414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f45c3-24b7-40db-b818-b4f905f3b931 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.458886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f45c3-24b7-40db-b818-b4f905f3b931 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.470081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 381/771] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:23.470081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da161203-6481-4be6-965f-2e4947dab7c6 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.470081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da161203-6481-4be6-965f-2e4947dab7c6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.470081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da161203-6481-4be6-965f-2e4947dab7c6 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.470081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 382/772] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:23.472858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da161203-6481-4be6-965f-2e4947dab7c6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.538756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:23.563636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 391/773] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:38:23.565046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a63b25-e991-46e0-8763-a86d0a572a6e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.565343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a63b25-e991-46e0-8763-a86d0a572a6e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.565622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a63b25-e991-46e0-8763-a86d0a572a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.582546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-13a63b25-e991-46e0-8763-a86d0a572a6e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:23.583582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-13a63b25-e991-46e0-8763-a86d0a572a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:23.602860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da161203-6481-4be6-965f-2e4947dab7c6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.645988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.647034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.647317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.647592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.648616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] RBAC: Adding query filter params (type=network) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:23.648929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.651063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.661676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 383/774] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:23.661676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.664631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 392/775] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:23.665711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a63b25-e991-46e0-8763-a86d0a572a6e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.717335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:23.785704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55251e07-0a14-4159-936f-83446b7d9d7e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.788294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a63b25-e991-46e0-8763-a86d0a572a6e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.796719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:23.824663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.825601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.825883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.826154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.827093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:23.828155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:23.828793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:23.829287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:23.830278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] RBAC: Adding query filter params (interface=public, service_id=4235048889a349c6b637d65775543e79, region_id=RegionOne) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:23.830792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.832432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.836644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:23.838148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:23.842798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 393/776] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/endpoints?service_id=4235048889a349c6b637d65775543e79&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:23.843860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:23.861627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:23.862603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:23.863537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:23.863537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:23.863537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:23.863537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:23.865794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 384/777] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:23.866857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:23.963107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:23.977059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd205ad-da87-4620-a791-8ca9ae548067 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:23.977199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2045a-dd97-4c14-badf-cfb06ab39d8c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.016440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.017295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.017576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.017847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.018555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.019438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.019933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.020452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.021334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.022911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.029170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 385/778] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:24.029313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:24.030358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:24.047695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:24.048748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:24.049617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:24.052997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 394/779] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:24.061387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.153908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:24.183248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4b4d7-ab07-434a-952a-3bfb275620dc None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.225495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.226190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.226525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.226803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.227554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:24.227871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.229631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.244031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 386/780] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:24.251935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.279191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:24.313785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b881d-3373-4ee0-ae76-192dd7664335 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.349590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.350623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.351121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.351588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.352799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:24.353328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.354947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.365633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 395/781] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:24.374965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.375253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec83d87-2e42-43bb-be1a-dd877ff8513a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.415469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.416322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.416657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.416936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.425408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.427126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:24.447504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:24.448581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:24.449482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:24.451963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 387/782] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:24.459407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.497047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:24.507143 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cdde4-3d59-4c8c-9aaa-bbb5e6d6fb6e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.568641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:24.593148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.594448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.594964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.595471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.595593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534215c-0f5d-4e00-9f0a-f42264df364f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.620326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.622168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.631392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.637093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.637093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.637093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.637093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:24.637093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.637093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:24.651788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:24.652780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:24.653681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:24.653681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:24.653681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:24.653681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:24.653916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 388/783] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:24.660793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.660923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 396/784] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:24.669242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.805713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8764-05a6-48f7-a1a7-85f4c9ddf1d5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.828169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad16e6a-e06d-43d1-b493-0fea07922360 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:24.845489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6bad0-9738-4f69-8b06-279cb3af5b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.846403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6bad0-9738-4f69-8b06-279cb3af5b11 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.846692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6bad0-9738-4f69-8b06-279cb3af5b11 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.846971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6bad0-9738-4f69-8b06-279cb3af5b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.848306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6bad0-9738-4f69-8b06-279cb3af5b11 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=180e36ec0b254ae1813b2892c3117d87) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:24.848627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6bad0-9738-4f69-8b06-279cb3af5b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.850363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6bad0-9738-4f69-8b06-279cb3af5b11 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.857403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 389/785] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/role_assignments?user.id=180e36ec0b254ae1813b2892c3117d87&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:24.860370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f52f38-c0f4-4c1b-bfa6-93053a356bb5 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.860903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f52f38-c0f4-4c1b-bfa6-93053a356bb5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.861402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f52f38-c0f4-4c1b-bfa6-93053a356bb5 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.863101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 390/786] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:24.864124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:24.871139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033ec4-6512-48db-af25-b79f89ca5f4b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.872706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033ec4-6512-48db-af25-b79f89ca5f4b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.873229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0033ec4-6512-48db-af25-b79f89ca5f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.893397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:24.894515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:24.895002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:24.895538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:24.896526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:24.897055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:24.901771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:24.916128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e0033ec4-6512-48db-af25-b79f89ca5f4b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:24.918073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e0033ec4-6512-48db-af25-b79f89ca5f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:24.927779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 397/787] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:24.933156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:24.953050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:25.003088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 391/788] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:25.040343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033ec4-6512-48db-af25-b79f89ca5f4b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.129068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c74d1-47d1-47de-8914-679d1ca2ab76 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.188145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0033ec4-6512-48db-af25-b79f89ca5f4b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.190257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2c5a7-cab3-46c6-9cff-f3ec1294b786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.191282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2c5a7-cab3-46c6-9cff-f3ec1294b786 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.191691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2c5a7-cab3-46c6-9cff-f3ec1294b786 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.192150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2c5a7-cab3-46c6-9cff-f3ec1294b786 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.201498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2c5a7-cab3-46c6-9cff-f3ec1294b786 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:25.203160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2c5a7-cab3-46c6-9cff-f3ec1294b786 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-65b2c5a7-cab3-46c6-9cff-f3ec1294b786 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:25.223381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:25.224446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:25.225397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:25.225934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 398/789] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:25.231723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e543-44eb-43aa-afd3-d001cd0f0161 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.231723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e543-44eb-43aa-afd3-d001cd0f0161 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.231723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3e543-44eb-43aa-afd3-d001cd0f0161 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.231723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 399/790] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:25.236433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:25.237308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e543-44eb-43aa-afd3-d001cd0f0161 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.251040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4b66d-6883-4d01-bdbe-22e8aba37e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.252934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b66d-6883-4d01-bdbe-22e8aba37e2a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.253772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b66d-6883-4d01-bdbe-22e8aba37e2a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.254598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b66d-6883-4d01-bdbe-22e8aba37e2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.255454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4b66d-6883-4d01-bdbe-22e8aba37e2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:25.257756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4b66d-6883-4d01-bdbe-22e8aba37e2a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-09c4b66d-6883-4d01-bdbe-22e8aba37e2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:25.276347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:25.277465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 15:38:25.278366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:25.282326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 392/791] 10.222.0.50 () {56 vars in 1129 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:25.283894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea729941-5875-4b21-b988-b811268fd3a7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.284426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea729941-5875-4b21-b988-b811268fd3a7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.284868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea729941-5875-4b21-b988-b811268fd3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.302284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ea729941-5875-4b21-b988-b811268fd3a7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:25.303552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ea729941-5875-4b21-b988-b811268fd3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:25.325390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:25.370776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3e543-44eb-43aa-afd3-d001cd0f0161 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.380161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 393/792] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:25.381081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea729941-5875-4b21-b988-b811268fd3a7 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.417172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.418244 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.418707 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.419181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.420329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:25.420856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:25.422510 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:25.447949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 400/793] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:25.449354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.494952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea729941-5875-4b21-b988-b811268fd3a7 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.519857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:25.532273 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.533037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.533428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.533771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.534555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:25.534939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:25.536434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:25.546229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 394/794] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:25.547094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.560464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adc3bf-3c79-476d-a179-4bd7308bb25b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.594197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:25.598713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.599768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.600256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.600708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.616546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:25.626877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:25.627828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:25.628713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:25.636631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 401/795] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:25.636631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.660461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf467b-d3b3-4993-b364-7ec402be1d0f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.695806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.696503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.696782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.697053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.708640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:25.710143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:25.727520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:25.728506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:25.729529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:25.729529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:25.729529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:25.729529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:25.733466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 395/796] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:25.734314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.775839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:25.777419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e104a3-39c0-4077-8b5e-eda374184fe2 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.825107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.826601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.827352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.827827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.829381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:25.830172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:25.832356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:25.839149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbbd76-6b26-4605-a378-ed8f3efa0d53 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.842534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 402/797] 10.222.0.50 () {56 vars in 1124 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:25.843420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.883866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.884924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.885895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.886396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.887964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:25.888584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:25.890403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:25.891261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:25.908024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 396/798] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:25.908024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:25.954527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727c2b1-fbba-4454-801b-d6bea49d9502 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:25.995795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:25.996853 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:25.997356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:25.997855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:25.998779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:25.999816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.001843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.019401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 403/799] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:26.020622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.029502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:26.038232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b5e53e-3bf1-4190-b34d-8dbebae28e58 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.075524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.076404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.076879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.077173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.078024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.079661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.107773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 397/800] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:26.108940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.148288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cb1ac-e99d-43fd-9b97-8bad2148b453 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.158607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:26.202520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.203772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.204450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.205226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.219091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.221029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:26.240174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:26.241195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:26.242087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:26.242087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:26.242087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:26.242087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:26.244581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 404/801] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:26.248716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e458b-98dd-475c-8313-dbcd016c3403 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.250615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.305078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.305901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.306193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.306460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.319541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.323698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:26.353186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:26.354217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:26.355131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:26.357454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 398/802] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:26.363934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.391168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:26.416511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a7227-eff8-4d0e-90d0-8f7e410ee78e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.460307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741144a-f233-46a7-9eb2-6db6910f287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.461488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741144a-f233-46a7-9eb2-6db6910f287b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.461684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741144a-f233-46a7-9eb2-6db6910f287b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.461958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741144a-f233-46a7-9eb2-6db6910f287b None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.462686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741144a-f233-46a7-9eb2-6db6910f287b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:26.463374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741144a-f233-46a7-9eb2-6db6910f287b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.465168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741144a-f233-46a7-9eb2-6db6910f287b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.481151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 405/803] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:26.482582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a912a-6db8-4900-9aa8-25991bfb2990 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.482872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a912a-6db8-4900-9aa8-25991bfb2990 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.483152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a912a-6db8-4900-9aa8-25991bfb2990 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.484747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 406/804] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:26.488843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a912a-6db8-4900-9aa8-25991bfb2990 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.495117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa907e-70f1-4a62-a871-1c876adbdfc4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.515528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:26.549838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cb38c-3390-4815-927e-02306449d2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.551728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cb38c-3390-4815-927e-02306449d2dd None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.552238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cb38c-3390-4815-927e-02306449d2dd None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.552692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cb38c-3390-4815-927e-02306449d2dd None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.553662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cb38c-3390-4815-927e-02306449d2dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:26.554148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cb38c-3390-4815-927e-02306449d2dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.555818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cb38c-3390-4815-927e-02306449d2dd None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.573519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 399/805] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:26.575363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8103c-a33e-49b3-a3d5-92ecf6213b21 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.576094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8103c-a33e-49b3-a3d5-92ecf6213b21 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.576569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8103c-a33e-49b3-a3d5-92ecf6213b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.595279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d6e8103c-a33e-49b3-a3d5-92ecf6213b21 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:26.596598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d6e8103c-a33e-49b3-a3d5-92ecf6213b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:26.626298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a912a-6db8-4900-9aa8-25991bfb2990 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.663120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.664118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.664562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.664833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.672980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.674465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.686136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 400/806] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:26.687258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8103c-a33e-49b3-a3d5-92ecf6213b21 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:26.691205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:26.692164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:26.693125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 407/807] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:26.699581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:26.699883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.784130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:26.819060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24dea-5c1d-41af-a44b-9eb99998d130 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.820367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8103c-a33e-49b3-a3d5-92ecf6213b21 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:26.859532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.860459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.860741 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.861019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2933f88027fc435eba3064124e2dd5d5/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.863016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=2933f88027fc435eba3064124e2dd5d5, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.864224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:26.864639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.865351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:26.865829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:26.866299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:26.868080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:26.868438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:26.869981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:26.890672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 401/808] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:26 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2933f88027fc435eba3064124e2dd5d5/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:26.896425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 408/809] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:26.896644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.898144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:26.974198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.042404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1dcbf-0e98-4365-bd80-cd7b08e6dce2 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.068039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c32c7f-f475-460a-87d5-ec82572ef626 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.084540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.091825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d06a13c-e9df-40ec-bad8-70ecee27d7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.092960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a13c-e9df-40ec-bad8-70ecee27d7e9 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.093739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a13c-e9df-40ec-bad8-70ecee27d7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.094215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d06a13c-e9df-40ec-bad8-70ecee27d7e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.117159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.117966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.118289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.118566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] PATH_INFO: `/v3/services/network` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.119223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:27.122776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:27.136442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:27.137505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:27.138448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 402/810] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/services/network => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:27.149100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:27.185101 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06a13c-e9df-40ec-bad8-70ecee27d7e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:27.187184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d06a13c-e9df-40ec-bad8-70ecee27d7e9 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:27.262439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e33ca-b4f8-4b4a-92cb-45f8f24d12d6 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.306148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf95fe-bdbb-41f5-bc34-2654306c570e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.306823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf95fe-bdbb-41f5-bc34-2654306c570e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.307135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf95fe-bdbb-41f5-bc34-2654306c570e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.307415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf95fe-bdbb-41f5-bc34-2654306c570e None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.308415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf95fe-bdbb-41f5-bc34-2654306c570e None admin] RBAC: Adding query filter params (name=network) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:27.308599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf95fe-bdbb-41f5-bc34-2654306c570e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:27.310072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf95fe-bdbb-41f5-bc34-2654306c570e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:27.312342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.315950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 403/811] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:27.321252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d542e-c45c-4496-ba26-51962bef648e None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.321962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d542e-c45c-4496-ba26-51962bef648e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.322825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d542e-c45c-4496-ba26-51962bef648e None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.325033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 404/812] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:27.326042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d542e-c45c-4496-ba26-51962bef648e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:27.326882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 409/813] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:38:27.333281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd138436-f4e8-46ce-bb45-d47829259718 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.333418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd138436-f4e8-46ce-bb45-d47829259718 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.333707 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd138436-f4e8-46ce-bb45-d47829259718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.352382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-bd138436-f4e8-46ce-bb45-d47829259718 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:27.353779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bd138436-f4e8-46ce-bb45-d47829259718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:27.381843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.431627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 410/814] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:27.432888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd138436-f4e8-46ce-bb45-d47829259718 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:27.469393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d542e-c45c-4496-ba26-51962bef648e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.510882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.511554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.511837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.512146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.512917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] RBAC: Adding query filter params (type=network) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:27.513168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:27.514567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:27.520694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 405/815] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:27.525950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:27.547452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd138436-f4e8-46ce-bb45-d47829259718 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.584783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.585991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.586452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.586939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.592090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.598332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:27.600057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:27.618622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:27.619679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:27.620611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:27.620611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:27.620611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:27.620611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:27.622907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 411/816] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:27.623852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:27.640208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.640832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb107a5-29a1-4c95-8621-3523b03a17c8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.678513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.679409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.679761 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.680129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.692260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:27.702267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:27.703234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:27.704530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:27.708076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 406/817] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:27.710906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:27.735313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b427691-e502-47d2-a5d1-8e340f1bba34 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.772059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.773207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.773789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.774264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.775212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:27.777091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:27.808972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 412/818] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:27.811717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:27.864419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ddfbf9-d894-4d5b-8a90-d361458f049c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:27.898525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.943730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:27.962113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:27.962855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:27.963321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:27.963765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:27.965287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:27.965610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:27.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:27.997705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 407/819] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:28.002343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.022366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e8c6f-b31a-4188-bb2e-0b24b5bf0512 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.069672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.070362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.070642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.070917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.071676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:28.072008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.073523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.086310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 413/820] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:28.091847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.144680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7928c-0935-4b95-8208-4ed6b8dd6328 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.197344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.198760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.199486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.199940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.210902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.213717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.219242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855aeff-6e33-47e4-af35-a0c2c2289b6d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:28.234249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:28.235249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:28.236292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:28.243628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 408/821] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:28.243628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.277228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:28.285019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.286465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.286968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.287533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.303798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:28.304837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.307744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:28.332032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:28.333226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:28.334139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:28.334139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:28.334139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:28.334139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:28.336062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 414/822] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:28.340912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.393399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfabb6-bc80-4eb6-8a94-f2bd25d31575 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.436404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c939da8-c21b-4707-9de9-48d0a3f68b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.437076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c939da8-c21b-4707-9de9-48d0a3f68b0d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.437356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c939da8-c21b-4707-9de9-48d0a3f68b0d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.437629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c939da8-c21b-4707-9de9-48d0a3f68b0d None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.438421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c939da8-c21b-4707-9de9-48d0a3f68b0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:28.439144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c939da8-c21b-4707-9de9-48d0a3f68b0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.440987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c939da8-c21b-4707-9de9-48d0a3f68b0d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.454226 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 409/823] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:28.455704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a5f22-0920-4cf0-ba71-e1fd7211b0c4 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.456215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a5f22-0920-4cf0-ba71-e1fd7211b0c4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.456678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a5f22-0920-4cf0-ba71-e1fd7211b0c4 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.461963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 410/824] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:28.461963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a5f22-0920-4cf0-ba71-e1fd7211b0c4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.469764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6fadc3-c4fb-4736-98f2-530c553ffe7d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.506204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d7698-fd4c-48cf-806e-75aa463393cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.506986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d7698-fd4c-48cf-806e-75aa463393cb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.507304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d7698-fd4c-48cf-806e-75aa463393cb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.507597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d7698-fd4c-48cf-806e-75aa463393cb None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.508636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d7698-fd4c-48cf-806e-75aa463393cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:28.508944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d7698-fd4c-48cf-806e-75aa463393cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.510638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d7698-fd4c-48cf-806e-75aa463393cb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.522228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 415/825] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:28.528205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29838511-94ba-49ae-a34e-9f190820ac4a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.528425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29838511-94ba-49ae-a34e-9f190820ac4a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.528759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29838511-94ba-49ae-a34e-9f190820ac4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.544958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-29838511-94ba-49ae-a34e-9f190820ac4a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:28.546159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-29838511-94ba-49ae-a34e-9f190820ac4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:28.561923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:28.571429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a5f22-0920-4cf0-ba71-e1fd7211b0c4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.596221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:28.610772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.611433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.611715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.612010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.612845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=4da1612f1c1740ceb2a918f0a8dd057b) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:28.613159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.614621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.625234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 411/826] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/role_assignments?user.id=4da1612f1c1740ceb2a918f0a8dd057b&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:28.628987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.648011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 416/827] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:28.668018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29838511-94ba-49ae-a34e-9f190820ac4a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.772980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff97b5-e07c-45c7-b10d-0bd49a06aac1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.792350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29838511-94ba-49ae-a34e-9f190820ac4a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.814154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.814154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.814455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.814692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.825134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.827692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.840531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:28.841345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:28.841627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:28.841898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:28.842565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:28.843945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:28.847786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:28.848844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:28.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 412/828] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:28.855318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:28.861188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:28.862225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:28.863292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 417/829] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:28.869641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:28.879786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:28.915764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:28.983504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e388a9b-1148-438e-81e7-be77a4a98875 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:28.992847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2678e-bfcf-4eda-9b30-9eb154261923 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.027964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.028651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.029282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.029394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.029599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.029856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.030261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.030703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:29.030918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.031027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.031948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:29.032413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.032525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.034056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.039744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 418/830] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:29.044944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.055547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 413/831] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Jun 30 15:38:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:29.057828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.163111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:29.163741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ff370-9e7b-4f14-8532-7f4a25b1c46a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.165552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:29.189599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7029253-9901-4e8c-beca-f2c8db652a23 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.202887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d1d84-231e-4a98-809c-49edc714f128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.203569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1d84-231e-4a98-809c-49edc714f128 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.203854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1d84-231e-4a98-809c-49edc714f128 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.204157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1d84-231e-4a98-809c-49edc714f128 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.204882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d1d84-231e-4a98-809c-49edc714f128 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:29.205196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d1d84-231e-4a98-809c-49edc714f128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.206577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d1d84-231e-4a98-809c-49edc714f128 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.211288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 419/832] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:29.212641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd3539-9ff7-43dc-bded-10e6aa49e8a2 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.212938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd3539-9ff7-43dc-bded-10e6aa49e8a2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.213276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd3539-9ff7-43dc-bded-10e6aa49e8a2 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.214735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 420/833] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:29.220544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09d16-a547-4885-81b7-d04cb525250f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.220836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09d16-a547-4885-81b7-d04cb525250f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.221118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b09d16-a547-4885-81b7-d04cb525250f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.235941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.237158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.237627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.238087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.241091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c4b09d16-a547-4885-81b7-d04cb525250f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:29.242454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c4b09d16-a547-4885-81b7-d04cb525250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:29.250707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.252758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:29.276953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:29.277922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:29.278844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:29.278844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:29.278844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:29.278844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:29.284606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 414/834] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:29.291042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.336664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 421/835] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:29.355783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b09d16-a547-4885-81b7-d04cb525250f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.412699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e428a-6c79-40af-a511-4c9f91e5232a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.444087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:29.453671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.454638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.454930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.455208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.455948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:29.456286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.458123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.470289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b09d16-a547-4885-81b7-d04cb525250f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.474277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 415/836] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:29.474760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:29.483439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.508705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.509626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.509902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.510230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.520938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:29.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:29.529734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:29.530601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 422/837] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:29.536090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.605493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014bab9-5b7c-48c0-a255-abd5d7dcd79d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.652129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.653214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.653501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.653776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.663362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.663835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de6bcc-e981-4c0b-875e-01ad720d1c11 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.664866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:29.687183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:29.688676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:29.690068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:29.696029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 416/838] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:29.696029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.707829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.708955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.709240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.709515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.710621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:29.710933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.712659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.723258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 423/839] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:29.729135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.756434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:29.762032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:29.826075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f5a20-2aad-4582-a1c6-36301b0a4c3e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.852563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15a62a-d581-4e27-a8ac-74979d0a8b56 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:29.876929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.877870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.878540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.878813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.879873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:29.880214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.882541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.889357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:29.890265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:29.890619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:29.890961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:29.896943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 417/840] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:29.904584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:29.907419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:29.908960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:29.927202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:29.928203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:29.929215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:29.929215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:29.929215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:29.929215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:29.933251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 424/841] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:29.934502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:30.018794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:30.037923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a80c-2205-459a-b099-01ea26fa557a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:30.049179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:30.078167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:30.079350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:30.079776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:30.080257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/180e36ec0b254ae1813b2892c3117d87/roles/fbd9e52b6be84e29999adebc31c75d72` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:30.097557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=180e36ec0b254ae1813b2892c3117d87, role_id=fbd9e52b6be84e29999adebc31c75d72)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:30.099370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:30.122138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 418/842] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:29 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/180e36ec0b254ae1813b2892c3117d87/roles/fbd9e52b6be84e29999adebc31c75d72 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:38:30.187971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b7708-e07f-4356-a399-3434e7af9377 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:30.227733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:30.228518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:30.228857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:30.229189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:30.229939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:30.230289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:30.231735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:30.249578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 425/843] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:30.251108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:30.372991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cab728-a861-4337-af55-9b7a663a00b7 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:30.412075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:30.413149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:30.414012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:30.414455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:30.422786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:30.424493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:30.439553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:30.440776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 15:38:30.441993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:30.444320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 419/844] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:30.452035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:30.552667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec6d258-6a20-45eb-8ff0-caa556ffa15f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:30.589653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:30.596866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac029c67-f1b8-43a3-938f-d43284956250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:30.597733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029c67-f1b8-43a3-938f-d43284956250 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:30.598188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029c67-f1b8-43a3-938f-d43284956250 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:30.598602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029c67-f1b8-43a3-938f-d43284956250 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:30.599601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac029c67-f1b8-43a3-938f-d43284956250 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:30.600077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac029c67-f1b8-43a3-938f-d43284956250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:30.601658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac029c67-f1b8-43a3-938f-d43284956250 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:30.620607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 426/845] 10.222.0.50 () {56 vars in 1135 bytes} [Thu Jun 30 15:38:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:30.629069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:30.652533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:30.741089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d28f3-5e0f-439d-9071-bc29d1247665 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:30.760293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f09d81-a476-4fc0-85a0-ef71f2e681c8 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:30.761425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f09d81-a476-4fc0-85a0-ef71f2e681c8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:30.762260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f09d81-a476-4fc0-85a0-ef71f2e681c8 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:30.764160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 427/846] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:30.770265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8e3ed-5289-427c-99c7-54da8996bbfe None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:30.770811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8e3ed-5289-427c-99c7-54da8996bbfe None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:30.771424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8e3ed-5289-427c-99c7-54da8996bbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:30.782422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:30.783074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:30.783357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:30.783627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:30.793679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-59b8e3ed-5289-427c-99c7-54da8996bbfe None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:30.795507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-59b8e3ed-5289-427c-99c7-54da8996bbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:30.860849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:30.862705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:30.884018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 428/847] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:30.891469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:30.903619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8e3ed-5289-427c-99c7-54da8996bbfe None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:30.997264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 420/848] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:38:30.998982 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.032012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8e3ed-5289-427c-99c7-54da8996bbfe None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.055099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:31.075405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.076574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.077085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.077556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.078507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.080226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:31.093290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:31.094280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:31.095151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:31.102992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 429/849] 10.222.0.50 () {56 vars in 1132 bytes} [Thu Jun 30 15:38:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:31.104746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.114162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c7132-35c0-4f9b-acb9-6189da098b15 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.132886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:31.150886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.151905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.152066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.152274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.162848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.164322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:31.183081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:31.184045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:31.184968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:31.184968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:31.184968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:31.184968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:31.189096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 421/850] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:31.189981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.221283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402f453-5b2a-46e6-ba49-0221646a90c8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.260440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.261547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.262033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.262589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.264145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:31.264772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.268431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.278890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 430/851] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:31.283374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.294576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b8075-3593-4c37-ba3e-459a9169da28 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.331839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1898a3-7043-411f-b37b-fac1310d3661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.332529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1898a3-7043-411f-b37b-fac1310d3661 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.334194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1898a3-7043-411f-b37b-fac1310d3661 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.334375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1898a3-7043-411f-b37b-fac1310d3661 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.335767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1898a3-7043-411f-b37b-fac1310d3661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:31.336140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1898a3-7043-411f-b37b-fac1310d3661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.338103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1898a3-7043-411f-b37b-fac1310d3661 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.339188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:31.351834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 422/852] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:31.353242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0d814-c7cb-419e-9d4e-6c48d87ade4b None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.353532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0d814-c7cb-419e-9d4e-6c48d87ade4b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.353843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0d814-c7cb-419e-9d4e-6c48d87ade4b None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.355296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 423/853] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:31.360158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0d814-c7cb-419e-9d4e-6c48d87ade4b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.394626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:31.402679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69072377-158a-4745-adda-d5d8cf4dff0c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.436118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e792c-db3a-420f-a5b3-f7053f2e6bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.436793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e792c-db3a-420f-a5b3-f7053f2e6bc7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.437089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e792c-db3a-420f-a5b3-f7053f2e6bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.437357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e792c-db3a-420f-a5b3-f7053f2e6bc7 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.438048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e792c-db3a-420f-a5b3-f7053f2e6bc7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:31.438359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e792c-db3a-420f-a5b3-f7053f2e6bc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.439738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e792c-db3a-420f-a5b3-f7053f2e6bc7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.444377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 431/854] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:31.445644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7306a9-f33a-4f76-bacd-81f4b0bc81fa None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.445930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7306a9-f33a-4f76-bacd-81f4b0bc81fa None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.446207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7306a9-f33a-4f76-bacd-81f4b0bc81fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.466385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-2d7306a9-f33a-4f76-bacd-81f4b0bc81fa None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:31.467722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2d7306a9-f33a-4f76-bacd-81f4b0bc81fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:31.486599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0d814-c7cb-419e-9d4e-6c48d87ade4b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.533587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.534425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.534781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.535132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.545995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.547592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.556459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 432/855] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:31.562976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:31.564144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:31.565289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:31.566593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 424/856] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:31.572107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7306a9-f33a-4f76-bacd-81f4b0bc81fa None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.578299 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.621863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:31.696397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:31.701443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7306a9-f33a-4f76-bacd-81f4b0bc81fa None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.705617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188916b-5632-4c68-bbdb-e82718ddede9 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.745934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.746613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.746893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.747166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.747900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.749336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.749446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dbe53-bb44-4202-88a0-a0f223e4899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.750090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dbe53-bb44-4202-88a0-a0f223e4899e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.750377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dbe53-bb44-4202-88a0-a0f223e4899e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.750656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dbe53-bb44-4202-88a0-a0f223e4899e None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.751725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808dbe53-bb44-4202-88a0-a0f223e4899e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:31.752057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808dbe53-bb44-4202-88a0-a0f223e4899e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.754021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808dbe53-bb44-4202-88a0-a0f223e4899e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.767203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 425/857] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:31 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:31.776788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 433/858] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:31.783505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:31.906833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e5f55-4ca1-4939-8017-02387fd0ab9a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:31.944885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa3ff7-6255-4959-9cbc-b19680f44527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:31.945618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3ff7-6255-4959-9cbc-b19680f44527 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:31.945901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3ff7-6255-4959-9cbc-b19680f44527 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:31.946177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3ff7-6255-4959-9cbc-b19680f44527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:31.947020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3ff7-6255-4959-9cbc-b19680f44527 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2933f88027fc435eba3064124e2dd5d5) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:31.947331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3ff7-6255-4959-9cbc-b19680f44527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:31.948939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3ff7-6255-4959-9cbc-b19680f44527 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:31.955654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 426/859] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:31 2022] GET /identity/v3/role_assignments?user.id=2933f88027fc435eba3064124e2dd5d5&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:32.327437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0ff43-8184-476d-9b8c-9c9b9069769c None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:32.327806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0ff43-8184-476d-9b8c-9c9b9069769c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:32.328018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0ff43-8184-476d-9b8c-9c9b9069769c None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:32.329566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 434/860] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:32.335051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8117-c4e3-42b5-9538-319b4e7fff6f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:32.335569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8117-c4e3-42b5-9538-319b4e7fff6f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:32.336089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f8117-c4e3-42b5-9538-319b4e7fff6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:32.353359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1a6f8117-c4e3-42b5-9538-319b4e7fff6f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:32.354935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1a6f8117-c4e3-42b5-9538-319b4e7fff6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:32.358098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:32.423655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 427/861] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:32.447535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0ff43-8184-476d-9b8c-9c9b9069769c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:32.548424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:32.552348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0ff43-8184-476d-9b8c-9c9b9069769c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:32.588342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:32.589190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:32.589505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:32.589778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:32.603933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:32.611612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:32.612598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:32.614493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 435/862] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:38:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:32.622875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8117-c4e3-42b5-9538-319b4e7fff6f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:32.730811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f8117-c4e3-42b5-9538-319b4e7fff6f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:32.765714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:32.766381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:32.766707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:32.766976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:32.767717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:32.768043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:32.769417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:32.778165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 428/863] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:38:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:32.783766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:32.891642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797fe23c-55ea-4d04-a955-bbdd041cd602 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:32.927154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:32.927957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:32.928264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:32.928537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:32.939655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:32.941168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:32.961073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:32.962145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:32.963379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:32.963379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:32.963379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:32.963379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:32.963379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 436/864] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:32.969726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:32.973046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:33.079129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a4afb-a2fd-4ce7-8c27-9e5fd5560bcf None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:33.115815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:33.116558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.116834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.117150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.117872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:33.118177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:33.119663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:33.132163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 429/865] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:33.141342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:33.161609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:33.250306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2be2f-14e3-4fc8-93e1-03651f16966b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:33.284629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:33.285458 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.285737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.286012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.293946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:33.295410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:33.310118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:33.311082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:33.312076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 437/866] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:38:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:33.317234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:33.384401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:33.424813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401633f-f753-45b8-a57e-58e8b6896e4e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:33.459153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b516fb7-e108-44b7-82a5-c9c6940f90fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:33.459823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b516fb7-e108-44b7-82a5-c9c6940f90fc None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.460124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b516fb7-e108-44b7-82a5-c9c6940f90fc None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.460394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b516fb7-e108-44b7-82a5-c9c6940f90fc None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.461233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b516fb7-e108-44b7-82a5-c9c6940f90fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:33.461537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b516fb7-e108-44b7-82a5-c9c6940f90fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:33.462974 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b516fb7-e108-44b7-82a5-c9c6940f90fc None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:33.481234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 430/867] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Jun 30 15:38:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:33.486744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:33.531811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac886c5d-05f8-46ea-b6a2-d637b0f3742e None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.532200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac886c5d-05f8-46ea-b6a2-d637b0f3742e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.532434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac886c5d-05f8-46ea-b6a2-d637b0f3742e None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.533927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 431/868] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:33.538967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74111553-4b91-4016-98e1-f2887ca62231 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.539249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74111553-4b91-4016-98e1-f2887ca62231 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.539523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74111553-4b91-4016-98e1-f2887ca62231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.558096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-74111553-4b91-4016-98e1-f2887ca62231 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:33.559407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-74111553-4b91-4016-98e1-f2887ca62231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:33.569850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:33.606436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19f876-7bee-48df-8264-34d961b9b20f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:33.637389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 432/869] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:33.653744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:33.654652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.655148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.655568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.660622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74111553-4b91-4016-98e1-f2887ca62231 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:33.721045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:33.732570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:33.734532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:33.773774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74111553-4b91-4016-98e1-f2887ca62231 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:33.808161 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:33.816434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63c7c8-415b-41e5-b6a1-23c5594b5558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:33.817406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63c7c8-415b-41e5-b6a1-23c5594b5558 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.817828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63c7c8-415b-41e5-b6a1-23c5594b5558 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.818279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63c7c8-415b-41e5-b6a1-23c5594b5558 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.819241 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63c7c8-415b-41e5-b6a1-23c5594b5558 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:33.820827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63c7c8-415b-41e5-b6a1-23c5594b5558 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:33.847065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 438/870] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:33 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:38:33.847281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 433/871] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:33 2022] POST /identity/v3/services => generated 269 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:33.848506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:33.946547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e1744-0a72-4600-8ac2-ec18423c91fa None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.946888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e1744-0a72-4600-8ac2-ec18423c91fa None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.947150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e1744-0a72-4600-8ac2-ec18423c91fa None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.952790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 434/872] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:33.954572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f189c6-e965-4b43-abb3-8f3d49978dd3 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:33.954867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f189c6-e965-4b43-abb3-8f3d49978dd3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:33.955149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f189c6-e965-4b43-abb3-8f3d49978dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:33.958168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61cfe0-a8c3-4da1-b2df-91e29884347e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:33.973134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-72f189c6-e965-4b43-abb3-8f3d49978dd3 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:33.974382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-72f189c6-e965-4b43-abb3-8f3d49978dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:34.016110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.017330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.017615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.017891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.034945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.036442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.058579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 435/873] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:34.059982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:34.060935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:34.061851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:34.061851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:34.061851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:34.061851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:34.064063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 439/874] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:34.067243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f189c6-e965-4b43-abb3-8f3d49978dd3 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.081803 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.109311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.112440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.173696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f189c6-e965-4b43-abb3-8f3d49978dd3 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.190441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8feb69a-cb9f-42ea-a8ca-c859f513164a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.210603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69d745-e44c-4af1-9842-f68e1cc907f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.211607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d745-e44c-4af1-9842-f68e1cc907f4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.212138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d745-e44c-4af1-9842-f68e1cc907f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.212619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69d745-e44c-4af1-9842-f68e1cc907f4 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.213478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69d745-e44c-4af1-9842-f68e1cc907f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:34.213913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69d745-e44c-4af1-9842-f68e1cc907f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.215224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69d745-e44c-4af1-9842-f68e1cc907f4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.225312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d20296-c390-42d9-b248-8fef33c9e3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.226082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20296-c390-42d9-b248-8fef33c9e3f9 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.226360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20296-c390-42d9-b248-8fef33c9e3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.226628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20296-c390-42d9-b248-8fef33c9e3f9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.227275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d20296-c390-42d9-b248-8fef33c9e3f9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.228781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d20296-c390-42d9-b248-8fef33c9e3f9 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.231902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 436/875] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:34.233217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf381-6385-47d3-9cfa-cc06194ed9e8 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.233508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf381-6385-47d3-9cfa-cc06194ed9e8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.233786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbf381-6385-47d3-9cfa-cc06194ed9e8 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.239083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 437/876] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:34.239083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbf381-6385-47d3-9cfa-cc06194ed9e8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-23d20296-c390-42d9-b248-8fef33c9e3f9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:34.240802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:34.241888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:34.242762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 440/877] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:34.243575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12be5d-dd02-43e2-8ff4-5d12c802f7c3 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.243575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12be5d-dd02-43e2-8ff4-5d12c802f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.243709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12be5d-dd02-43e2-8ff4-5d12c802f7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.265931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-de12be5d-dd02-43e2-8ff4-5d12c802f7c3 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:34.267263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-de12be5d-dd02-43e2-8ff4-5d12c802f7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:34.348288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 441/878] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:34.349096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12be5d-dd02-43e2-8ff4-5d12c802f7c3 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.349229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.353483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbf381-6385-47d3-9cfa-cc06194ed9e8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.374823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.392078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.392968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.393361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.393690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.401891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.404004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:34.418890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:34.419837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:34.420864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 438/879] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:34.421369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.472288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12be5d-dd02-43e2-8ff4-5d12c802f7c3 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.506242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.506830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.507109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.507382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.508475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:34.508784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.510187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.514868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 442/880] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:34.515637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.532916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bc541-2abc-4a40-9314-2f9d0aa8e867 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.569093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.570451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.570883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.571319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.571983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.573641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:34.586269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:34.587308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:34.588239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:34.592414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 439/881] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:34.593262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.593836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.617239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b082c0-14ee-473a-9451-cb2ffd800405 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.621161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.656821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.657491 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.657770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.658036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.658890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:34.659201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.660670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.672427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 443/882] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:34.673263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.703804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c3980-2b69-45c9-8ced-232a1bdc93ff None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.742276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.743049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.743334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.743603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.744336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:34.744740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.746027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.751231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 440/883] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:34.752934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.786720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305da81-830f-4b6d-b5d1-4b7fa61a6d72 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.820906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.821584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.821865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.822140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.822945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:34.823251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.824674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.833864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 444/884] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:34.833864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.833864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.855192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:34.869430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a119fd-9509-45a4-b4ab-1eb2ade59512 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.906595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:34.907358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:34.907639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:34.907912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:34.908766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=180e36ec0b254ae1813b2892c3117d87) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:34.909075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:34.910504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:34.920148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 441/885] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/role_assignments?user.id=180e36ec0b254ae1813b2892c3117d87&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:34.920148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:34.958653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e5445-fba3-4414-947d-efb9c3d50d5f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:34.999493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:35.000579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:35.001124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:35.001574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:35.002842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] RBAC: Adding query filter params (interface=public, service_id=0758c9557b21480ca4480b0774bde9f1, region_id=RegionOne) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:35.003383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:35.005344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:35.016482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 445/886] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/endpoints?service_id=0758c9557b21480ca4480b0774bde9f1&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:35.018959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:35.063389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c878485-c69f-485e-a3f2-c39841311b8a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:35.102596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:35.114372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a604d52-c2a4-48bb-85c1-26b79e9eccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:35.115403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604d52-c2a4-48bb-85c1-26b79e9eccce None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:35.116065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604d52-c2a4-48bb-85c1-26b79e9eccce None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:35.116065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604d52-c2a4-48bb-85c1-26b79e9eccce None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:35.116723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a604d52-c2a4-48bb-85c1-26b79e9eccce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:35.117029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a604d52-c2a4-48bb-85c1-26b79e9eccce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:35.118409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a604d52-c2a4-48bb-85c1-26b79e9eccce None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:35.124693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 442/887] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:35.148573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66620767-3d50-42e3-b4f2-5b2d7ea98f25 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:35.193568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898ad9d-1bfd-48ad-908b-590bb5195dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:35.194307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898ad9d-1bfd-48ad-908b-590bb5195dfa None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:35.194590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898ad9d-1bfd-48ad-908b-590bb5195dfa None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:35.194914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898ad9d-1bfd-48ad-908b-590bb5195dfa None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:35.195629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898ad9d-1bfd-48ad-908b-590bb5195dfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:35.197102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898ad9d-1bfd-48ad-908b-590bb5195dfa None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:35.204354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 446/888] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:35 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:38:35.977536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dceec-651d-4635-aa53-14c7ef4138e2 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:35.978077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dceec-651d-4635-aa53-14c7ef4138e2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:35.978077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825dceec-651d-4635-aa53-14c7ef4138e2 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:35.979821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 443/889] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:35.985267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78e3fa-1ceb-4ce7-b99d-b629e409dd48 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:35.985673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78e3fa-1ceb-4ce7-b99d-b629e409dd48 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:35.985843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78e3fa-1ceb-4ce7-b99d-b629e409dd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:36.003504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8b78e3fa-1ceb-4ce7-b99d-b629e409dd48 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:36.004750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8b78e3fa-1ceb-4ce7-b99d-b629e409dd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:36.076100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 447/890] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:36.095083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dceec-651d-4635-aa53-14c7ef4138e2 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:36.116914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:36.209027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825dceec-651d-4635-aa53-14c7ef4138e2 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:36.243911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:36.244727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:36.245017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:36.245284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:36.246019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:36.247475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:36.261353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:36.262458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:36.263330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 444/891] 10.222.0.50 () {56 vars in 1132 bytes} [Thu Jun 30 15:38:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:36.269106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78e3fa-1ceb-4ce7-b99d-b629e409dd48 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:36.331732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:36.375389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78e3fa-1ceb-4ce7-b99d-b629e409dd48 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:36.414971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:36.415642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:36.415922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:36.416276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:36.416988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:36.417294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:36.418675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:36.428059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 448/892] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:36.429416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:36.516087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:36.536096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ae8e4-3c10-45a4-b324-d2e1a07b3df2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:36.569899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:36.570577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:36.570860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:36.571145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:36.571855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:36.572189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:36.573616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:36.578969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 445/893] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:36.584745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:36.692566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f59e56-54fd-4a3c-bb54-46b052a2f21e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:36.720365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:36.727627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ad301-7bb5-4aae-a394-8e68d571749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:36.728325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad301-7bb5-4aae-a394-8e68d571749b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:36.728671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad301-7bb5-4aae-a394-8e68d571749b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:36.728944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ad301-7bb5-4aae-a394-8e68d571749b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:36.729823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ad301-7bb5-4aae-a394-8e68d571749b None admin] RBAC: Adding query filter params (interface=public, service_id=512adf73b8dd47ca8d619cb23150c6df, region_id=RegionOne) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:36.730484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ad301-7bb5-4aae-a394-8e68d571749b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:36.732081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ad301-7bb5-4aae-a394-8e68d571749b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:36.742164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 449/894] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:38:36 2022] GET /identity/v3/endpoints?service_id=512adf73b8dd47ca8d619cb23150c6df&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:36.743681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:36.846054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38039cea-9c4b-4427-a78a-76f339f6ed2c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:36.880389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e673fb-4333-411f-a0ea-47bfd035a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:36.881062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e673fb-4333-411f-a0ea-47bfd035a0eb None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:36.881338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e673fb-4333-411f-a0ea-47bfd035a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:36.881606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e673fb-4333-411f-a0ea-47bfd035a0eb None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:36.882293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e673fb-4333-411f-a0ea-47bfd035a0eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:36.884059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e673fb-4333-411f-a0ea-47bfd035a0eb None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:36.896219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 446/895] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:36 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:38:37.164006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae40f-3356-4558-ab2d-a5aeaa347d57 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.164545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae40f-3356-4558-ab2d-a5aeaa347d57 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.164545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae40f-3356-4558-ab2d-a5aeaa347d57 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.166188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 450/896] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:37.171610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3be87-a0be-46db-b143-5adb9dc1de5e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.172108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3be87-a0be-46db-b143-5adb9dc1de5e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.172188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3be87-a0be-46db-b143-5adb9dc1de5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.185285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a0b1d-70a5-4778-bcc1-200d2abe2eb0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.185608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a0b1d-70a5-4778-bcc1-200d2abe2eb0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.185832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a0b1d-70a5-4778-bcc1-200d2abe2eb0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.187451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 451/897] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:37.191073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-12e3be87-a0be-46db-b143-5adb9dc1de5e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:37.192319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-12e3be87-a0be-46db-b143-5adb9dc1de5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:37.192695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a58442-77f6-4629-9758-09bf81072b0d None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.193176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a58442-77f6-4629-9758-09bf81072b0d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.193664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a58442-77f6-4629-9758-09bf81072b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.194403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:37.210968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-29a58442-77f6-4629-9758-09bf81072b0d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:37.212519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-29a58442-77f6-4629-9758-09bf81072b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:37.263388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 447/898] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:37.286725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3be87-a0be-46db-b143-5adb9dc1de5e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.290909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 452/899] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:37.313556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a58442-77f6-4629-9758-09bf81072b0d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.390325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3be87-a0be-46db-b143-5adb9dc1de5e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:37.420311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a58442-77f6-4629-9758-09bf81072b0d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:37.425576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868be8e8-50de-46c4-a94b-8599bb733949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:37.426409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868be8e8-50de-46c4-a94b-8599bb733949 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.426681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868be8e8-50de-46c4-a94b-8599bb733949 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.426953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868be8e8-50de-46c4-a94b-8599bb733949 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.434291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:37.438164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:37.439810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868be8e8-50de-46c4-a94b-8599bb733949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-868be8e8-50de-46c4-a94b-8599bb733949 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:37.448865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:37.450100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:37.450997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 448/900] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Jun 30 15:38:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:37.451720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187599d-e8c9-47ad-acac-c090da5d27bb None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.451940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187599d-e8c9-47ad-acac-c090da5d27bb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.452237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187599d-e8c9-47ad-acac-c090da5d27bb None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.456032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 449/901] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:37.456513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187599d-e8c9-47ad-acac-c090da5d27bb None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.465622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc34df-728c-43d3-a6e0-f3e3c8aa1e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:37.466675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc34df-728c-43d3-a6e0-f3e3c8aa1e65 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.467168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc34df-728c-43d3-a6e0-f3e3c8aa1e65 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.467641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc34df-728c-43d3-a6e0-f3e3c8aa1e65 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.468723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc34df-728c-43d3-a6e0-f3e3c8aa1e65 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:37.470534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc34df-728c-43d3-a6e0-f3e3c8aa1e65 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:37.487403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 453/902] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:37 2022] POST /identity/v3/services => generated 241 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:37.489020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f8564-5aa0-4605-ace7-1fd30e032c7e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.489376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f8564-5aa0-4605-ace7-1fd30e032c7e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.489720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f8564-5aa0-4605-ace7-1fd30e032c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.517379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-272f8564-5aa0-4605-ace7-1fd30e032c7e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:37.519048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-272f8564-5aa0-4605-ace7-1fd30e032c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:37.601692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187599d-e8c9-47ad-acac-c090da5d27bb None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:37.610236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 454/903] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:37.634061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f8564-5aa0-4605-ace7-1fd30e032c7e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.642284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:37.643366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.643843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.644344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.645302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:37.645847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:37.647469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:37.657575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 450/904] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Jun 30 15:38:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:37.664916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.734062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:37.763924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f8564-5aa0-4605-ace7-1fd30e032c7e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:37.774195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:37.795378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122160f-14e9-4f8b-b052-6e8d5cc6ab3c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:37.806460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:37.807758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.808218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.808663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.809594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:37.811743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:37.824662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:37.825904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 15:38:37.826962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:37.829460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 455/905] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:37.830858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:37.831658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.832004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.832330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.834851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.844427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:37.845734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:37.863666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:37.864657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:37.865582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:37.865582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:37.865582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:37.865582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:37.869808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 451/906] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:37.870952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.941370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b5d7c-9a8a-45e0-a2b0-1e846f098782 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:37.977634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:37.978801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:37.979283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:37.979771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:37.980722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:37.981253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:37.982829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:37.984841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8938d8-7087-4202-9f52-09d25a4e6be8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:37.988849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 456/907] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:37.993959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:37.995589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:38.036213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:38.037043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.037329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.037606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.038370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:38.038683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:38.040418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:38.048425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:38.060071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 452/908] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:38.064674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:38.118212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ef6c5-da4d-4ef1-9b93-18b1f8bd2029 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:38.153091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:38.153763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.154046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.154323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.155035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:38.155345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:38.156952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:38.166930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 457/909] 10.222.0.50 () {56 vars in 1114 bytes} [Thu Jun 30 15:38:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:38.168107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:38.182437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e318d4c-494b-4cde-b423-d185169887c2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:38.226742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:38.228034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.228561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.228934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.237027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:38.238769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:38.255264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:38.256362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:38.257338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:38.262104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 453/910] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:38:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:38.263051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:38.294476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:38.297849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f2f72-30dd-4013-83e2-bc12d807912b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:38.301079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:38.335387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:38.336485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.336974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.337470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.338477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:38.340094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:38.360600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 458/911] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:38.387566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd76d5-19e9-4201-b7b4-c455e0aacd1c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:38.432325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:38.433012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.433290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.433559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.434556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:38.434866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:38.436439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:38.457311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 454/912] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Jun 30 15:38:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:38.463279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:38.575724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea37c-005f-4437-9e74-c9f45fe57bb7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:38.612165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:38.613190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.613700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.614163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.640345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:38.687329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:38.688842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:38.790527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 459/913] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:38 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:38:38.796342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:38.893276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:38.900986 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448800a-e62a-4a2f-9665-332a0f56d248 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:38.935495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07bf16-7e17-400c-832b-c0ccb8edd077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:38.936333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07bf16-7e17-400c-832b-c0ccb8edd077 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.936611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07bf16-7e17-400c-832b-c0ccb8edd077 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.936877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07bf16-7e17-400c-832b-c0ccb8edd077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.947468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07bf16-7e17-400c-832b-c0ccb8edd077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:38.948961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07bf16-7e17-400c-832b-c0ccb8edd077 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-dd07bf16-7e17-400c-832b-c0ccb8edd077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:38.967031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:38.968037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:38.970904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:38.970904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:38.970904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:38.970904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:38.970904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 455/914] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:38.974225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:38.977510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40447484-2a8d-4040-87da-10e569464bc3 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.977852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40447484-2a8d-4040-87da-10e569464bc3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.978134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40447484-2a8d-4040-87da-10e569464bc3 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:38.979836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 456/915] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:38.985004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc4480-5906-44b7-9e7a-41e13f4bae4c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:38.985290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc4480-5906-44b7-9e7a-41e13f4bae4c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:38.985570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc4480-5906-44b7-9e7a-41e13f4bae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.002463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-b1cc4480-5906-44b7-9e7a-41e13f4bae4c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:39.003656 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b1cc4480-5906-44b7-9e7a-41e13f4bae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:39.062141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:39.080337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 457/916] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:39.089746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56410c53-1202-4c9e-9cae-d54c5559345e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.098225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc4480-5906-44b7-9e7a-41e13f4bae4c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.132580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.133839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.135044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.135327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.136075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:39.136380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.137761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.156167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 460/917] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:39.158685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.169856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:39.211014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc4480-5906-44b7-9e7a-41e13f4bae4c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.260047 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.261221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.262215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.262715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.263710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.266005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:39.285385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:39.286899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 15:38:39.288172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:39.291497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 458/918] 10.222.0.50 () {56 vars in 1132 bytes} [Thu Jun 30 15:38:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:39.297417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a31c-abd0-4433-9f36-5e3a579d530e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.297917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.313815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:39.334018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.335058 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.335578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.336021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.346585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.348199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:39.362555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:39.363765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:39.364951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:39.366522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 461/919] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:39.371810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.423443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29ef3-7690-4f8a-8b26-992b9fbc6937 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.451211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:39.464363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.465769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.466527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.467009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.468218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:39.469175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.471301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.479011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 459/920] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:39.486603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.493933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe353185-36d6-4747-af9f-515742d91ca2 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.532212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.532938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.533218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.533489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.534320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:39.534635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.536099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.553314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 462/921] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:39.554497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.598825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:39.607698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702fc09-5859-43b2-a60f-55481d4d4961 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.646439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0b8d8-d418-4072-ab2a-4dc268c87434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.647214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b8d8-d418-4072-ab2a-4dc268c87434 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.647498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b8d8-d418-4072-ab2a-4dc268c87434 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.647777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b8d8-d418-4072-ab2a-4dc268c87434 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.648509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0b8d8-d418-4072-ab2a-4dc268c87434 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:39.648816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0b8d8-d418-4072-ab2a-4dc268c87434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.650428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0b8d8-d418-4072-ab2a-4dc268c87434 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.660315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 460/922] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:39.660315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99f792-9363-417b-9401-9d79e84349f6 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.660315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99f792-9363-417b-9401-9d79e84349f6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.660315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99f792-9363-417b-9401-9d79e84349f6 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.660315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 461/923] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:39.663516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99f792-9363-417b-9401-9d79e84349f6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.679807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1b3ab-e963-4b4f-9924-a28fadcf259b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.715206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523e84-6a72-4483-b2c1-b6ee5e5228ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.715867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523e84-6a72-4483-b2c1-b6ee5e5228ad None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.716187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523e84-6a72-4483-b2c1-b6ee5e5228ad None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.716452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523e84-6a72-4483-b2c1-b6ee5e5228ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.717285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e523e84-6a72-4483-b2c1-b6ee5e5228ad None admin] RBAC: Adding query filter params (role.id=4ab959c8c5514faa9bc905b24a221755, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=180e36ec0b254ae1813b2892c3117d87) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:39.717596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e523e84-6a72-4483-b2c1-b6ee5e5228ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.719042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e523e84-6a72-4483-b2c1-b6ee5e5228ad None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.723326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:39.732022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 463/924] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:39 2022] GET /identity/v3/role_assignments?user.id=180e36ec0b254ae1813b2892c3117d87&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=4ab959c8c5514faa9bc905b24a221755 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:39.732022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b44f5-2741-43bf-90a8-1066128b038c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.732022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b44f5-2741-43bf-90a8-1066128b038c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.732022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b44f5-2741-43bf-90a8-1066128b038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.749432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d44b44f5-2741-43bf-90a8-1066128b038c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:39.751578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d44b44f5-2741-43bf-90a8-1066128b038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:39.791848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99f792-9363-417b-9401-9d79e84349f6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:39.835909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bf517-b172-44db-900f-03a194311560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:39.837044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 464/925] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:39.844143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bf517-b172-44db-900f-03a194311560 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:39.844592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bf517-b172-44db-900f-03a194311560 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:39.852066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bf517-b172-44db-900f-03a194311560 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:39.852066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bf517-b172-44db-900f-03a194311560 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:39.852066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bf517-b172-44db-900f-03a194311560 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:39.863767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b44f5-2741-43bf-90a8-1066128b038c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:39.877016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 462/926] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:39.899445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:39.979703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b44f5-2741-43bf-90a8-1066128b038c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:40.025630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:40.026471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.026746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.027013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.027674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:40.029333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:40.042861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:40.044131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.045052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 465/927] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:40.052664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bf517-b172-44db-900f-03a194311560 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:40.156398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bf517-b172-44db-900f-03a194311560 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:40.169862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:40.191228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:40.191908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.192235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.192586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.193460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:40.193886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:40.195691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:40.201563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 463/928] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:40.207168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:40.302868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:40.313055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d00418-d1f5-46ec-a693-9b5ab2e509c7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:40.347223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:40.348012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.348288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.348561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.349278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:40.349579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:40.351091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:40.356608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 466/929] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:40.362524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:40.469904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0d526-c6de-43fe-8c21-f9b2adec673b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:40.505203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62f134-f60c-4752-bba6-abd086f19996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:40.505872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f134-f60c-4752-bba6-abd086f19996 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.506155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f134-f60c-4752-bba6-abd086f19996 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.506431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62f134-f60c-4752-bba6-abd086f19996 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.507235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f134-f60c-4752-bba6-abd086f19996 None admin] RBAC: Adding query filter params (interface=public, service_id=b57abaeb507e4a148746ded344573b31, region_id=RegionOne) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:40.507547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f134-f60c-4752-bba6-abd086f19996 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:40.509319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62f134-f60c-4752-bba6-abd086f19996 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:40.515193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 464/930] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:38:40 2022] GET /identity/v3/endpoints?service_id=b57abaeb507e4a148746ded344573b31&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:40.520826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:40.582105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d9a3-b4ce-4748-af07-e5c5c47dd48b None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.582441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d9a3-b4ce-4748-af07-e5c5c47dd48b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.582668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2d9a3-b4ce-4748-af07-e5c5c47dd48b None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.584204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 465/931] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:40.590708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bdaa2-d71d-4e57-a8b9-6909d011504e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.590997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bdaa2-d71d-4e57-a8b9-6909d011504e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.591276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bdaa2-d71d-4e57-a8b9-6909d011504e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.608747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e87bdaa2-d71d-4e57-a8b9-6909d011504e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:40.609960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e87bdaa2-d71d-4e57-a8b9-6909d011504e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:40.625826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61eaab-4d40-468f-b575-b0c15ebeae33 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:40.646688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:40.663890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:40.664809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.665175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.665536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.666517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:40.668519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:40.675744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 467/932] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:40 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:38:40.684249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 466/933] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:40.715390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:40.737392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:40.827951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b71f69-22d5-4d4d-a962-aa6d32931fe5 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:40.862816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:40.863490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:40.863772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:40.864079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:40.864868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:40.866319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:40.867707 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING keystone.server.flask.common [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 15:38:40.889416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.sql.core [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 15:38:40.889416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 15:38:40.889416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [parameters: {'id': 'ff10dc2633a047ffb58b23ec2a026828', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 15:38:40.889416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91958) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ff10dc2633a047ffb58b23ec2a026828.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ff10dc2633a047ffb58b23ec2a026828. Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application conn.query(q) Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 15:38:40.907282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application result.read() Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 15:38:40.908237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application next(self.gen) Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application yield resource Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application next(self.gen) Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise self.value Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application session.commit() Jun 30 15:38:40.909446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self.session.flush() Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application compat.raise_( Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 15:38:40.910342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 15:38:40.911251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application conn.query(q) Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application result.read() Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 15:38:40.912715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [parameters: {'id': 'ff10dc2633a047ffb58b23ec2a026828', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:40.914252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:40.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ff10dc2633a047ffb58b23ec2a026828. Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:40.922030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 468/934] 10.222.0.50 () {60 vars in 1140 bytes} [Thu Jun 30 15:38:40 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:40.927364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bdaa2-d71d-4e57-a8b9-6909d011504e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:41.032190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bdaa2-d71d-4e57-a8b9-6909d011504e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:41.067676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5fac3-913c-42e1-bc80-8fb2797c1816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:41.068864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5fac3-913c-42e1-bc80-8fb2797c1816 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:41.069377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5fac3-913c-42e1-bc80-8fb2797c1816 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:41.069850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5fac3-913c-42e1-bc80-8fb2797c1816 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:41.078130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5fac3-913c-42e1-bc80-8fb2797c1816 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:41.079642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef5fac3-913c-42e1-bc80-8fb2797c1816 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-aef5fac3-913c-42e1-bc80-8fb2797c1816 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:41.093539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:41.094479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:41.095653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 467/935] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:41.103971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:41.186136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:41.215490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961d2e3-4429-4b59-8820-b1f743b9f5eb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:41.250002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e01d-0928-440e-bd85-402a02dea6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:41.252393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e01d-0928-440e-bd85-402a02dea6c8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:41.252880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e01d-0928-440e-bd85-402a02dea6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:41.253372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e01d-0928-440e-bd85-402a02dea6c8 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:41.254346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e01d-0928-440e-bd85-402a02dea6c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:41.254878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e01d-0928-440e-bd85-402a02dea6c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:41.256750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e01d-0928-440e-bd85-402a02dea6c8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:41.296106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 469/936] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:41.810234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91cb16-5fab-4ddd-8bcb-32aa27fd3d15 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:41.810820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91cb16-5fab-4ddd-8bcb-32aa27fd3d15 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:41.810820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91cb16-5fab-4ddd-8bcb-32aa27fd3d15 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:41.812472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 468/937] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:41.818247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe52aef-07a0-4b0b-a7a3-c3152a999ac9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:41.818669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe52aef-07a0-4b0b-a7a3-c3152a999ac9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:41.818860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe52aef-07a0-4b0b-a7a3-c3152a999ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:41.839055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-cfe52aef-07a0-4b0b-a7a3-c3152a999ac9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:41.840665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-cfe52aef-07a0-4b0b-a7a3-c3152a999ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:41.913330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 470/938] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:41.935379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91cb16-5fab-4ddd-8bcb-32aa27fd3d15 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:41.966072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:42.039534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91cb16-5fab-4ddd-8bcb-32aa27fd3d15 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.069471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7eee8-7efc-4350-8c8a-fc1c672b248f None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.069680 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7eee8-7efc-4350-8c8a-fc1c672b248f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.069932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7eee8-7efc-4350-8c8a-fc1c672b248f None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.071658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 471/939] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:42.076980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761236e5-6924-4951-aef5-8471a814fb60 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.077373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761236e5-6924-4951-aef5-8471a814fb60 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.077652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761236e5-6924-4951-aef5-8471a814fb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.077812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.078641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.078929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.079205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.095639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:42.097920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-761236e5-6924-4951-aef5-8471a814fb60 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:42.099117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-761236e5-6924-4951-aef5-8471a814fb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:42.107413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:42.108580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:42.109610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 469/940] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Jun 30 15:38:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:42.114591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.170118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:42.181942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 472/941] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:42.208408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761236e5-6924-4951-aef5-8471a814fb60 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.227176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6ed7c-2ed4-41ae-8916-b06a9b960440 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.236339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:42.262174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.262853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.263133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.263406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.264200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:42.264511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:42.266429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:42.284834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 470/942] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:42.287612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.326997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761236e5-6924-4951-aef5-8471a814fb60 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.366971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.368265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.368784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.369287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] PATH_INFO: `/v3/services/compute` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.370185 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:42.372082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:42.387518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:42.388633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:42.389468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:42.392140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 473/943] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:42.395468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.426885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ca910-a5ed-4018-abd5-543521b93032 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.463913 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:42.479638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.480498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.480800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.481071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.496104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:42.497732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:42.513888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d71f4-7c87-4610-b23f-abc16f68b99d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:42.518069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:42.518992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:42.519917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:42.519917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:42.519917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:42.519917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:42.524450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 471/944] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:42.526532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.530960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:42.558510 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.559404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.559774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.560175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.560993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:42.565091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:42.565091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:42.568782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 474/945] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:42.574094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.659773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bdf323-2e30-4b1f-aed0-30ca093d157e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.700132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c3de9f-8d38-42ff-a742-6510e978b6a8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.700341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.700953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.701324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.701599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.702342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:42.702646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:42.704064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:42.727086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 472/946] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:42.727964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:42.728775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.740104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8a7aca-5234-453e-9b3e-6b4d67e32eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.740819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7aca-5234-453e-9b3e-6b4d67e32eba None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.741097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7aca-5234-453e-9b3e-6b4d67e32eba None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.741371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a7aca-5234-453e-9b3e-6b4d67e32eba None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.742087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a7aca-5234-453e-9b3e-6b4d67e32eba None admin] RBAC: Adding query filter params (type=compute) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:42.742392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a7aca-5234-453e-9b3e-6b4d67e32eba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:42.743772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8a7aca-5234-453e-9b3e-6b4d67e32eba None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:42.749092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 475/947] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:42.784761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dfc97-494e-45e4-9ebc-405e9e2978bb None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.785084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dfc97-494e-45e4-9ebc-405e9e2978bb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.785308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50dfc97-494e-45e4-9ebc-405e9e2978bb None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.787025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 476/948] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:42.794466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2363e-810b-4a45-b74c-91e9f7be4d00 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.799521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2363e-810b-4a45-b74c-91e9f7be4d00 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.799521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2363e-810b-4a45-b74c-91e9f7be4d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.809940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:42.814984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ffd2363e-810b-4a45-b74c-91e9f7be4d00 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:42.816465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ffd2363e-810b-4a45-b74c-91e9f7be4d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:42.859577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d8e7b2-8b29-46b8-bd8f-631a59a05000 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:42.901742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:42.902636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:42.902952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:42.903222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:42.904419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 477/949] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:42.911783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:42.913289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:42.922307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2363e-810b-4a45-b74c-91e9f7be4d00 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:42.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:42.933527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:42.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 473/950] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:42.939185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.049011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2363e-810b-4a45-b74c-91e9f7be4d00 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.073539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:43.074605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956347b-c29c-441c-8f65-c478c1fc6e9e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.102226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.103554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.103795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:43.104166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.104920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.106047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.108222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.117845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.118520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.118821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.119096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.119940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:43.120273 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.121716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:43.126230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:43.127460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:43.128457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:43.131079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 478/951] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:43.138020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.152572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 474/952] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Jun 30 15:38:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:43.155383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.262646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433f2b6-023c-4ee3-8799-a7abb4a3fe81 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.300863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.301546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.301825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.302098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.302808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:43.303109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.304520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.305735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19152fb-7fc1-4719-a277-d13642357c23 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.314533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 479/953] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:43.315440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.342375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.343524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.343967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.344458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.356018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.357753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.361239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:43.378061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:43.380310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:43.382931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:43.382931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:43.382931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:43.382931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:43.384676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 475/954] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:43.388105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.417941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f58e8-6eaf-4355-ae9a-75f127355a28 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.424966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:43.454671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.455338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.455615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.455885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.456774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:43.457021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.458490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.469790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 480/955] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:43.470938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.504414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c268c-ed1a-4697-b8bc-f2fc27f52249 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.542377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a7aa2-78c6-4bc8-ac2d-b5938e192d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.543380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7aa2-78c6-4bc8-ac2d-b5938e192d49 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.544071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7aa2-78c6-4bc8-ac2d-b5938e192d49 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.544830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7aa2-78c6-4bc8-ac2d-b5938e192d49 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.545907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a7aa2-78c6-4bc8-ac2d-b5938e192d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:43.546416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a7aa2-78c6-4bc8-ac2d-b5938e192d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.548096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a7aa2-78c6-4bc8-ac2d-b5938e192d49 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.564028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 476/956] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:43.564028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260657-a25c-4be5-bf22-a9526aa4b494 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.564028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260657-a25c-4be5-bf22-a9526aa4b494 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.564028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b260657-a25c-4be5-bf22-a9526aa4b494 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.564418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 477/957] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:43.568446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260657-a25c-4be5-bf22-a9526aa4b494 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.582355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c8e1a-8593-4bbf-a13b-a58e3288b814 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.592188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:43.619785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca6da2-8942-4b33-810b-047294afdb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.620476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca6da2-8942-4b33-810b-047294afdb3e None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.620755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca6da2-8942-4b33-810b-047294afdb3e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.621028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca6da2-8942-4b33-810b-047294afdb3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.621767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca6da2-8942-4b33-810b-047294afdb3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.623155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca6da2-8942-4b33-810b-047294afdb3e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.644418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 481/958] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:43.645871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ae61f-e7b4-45d1-954b-7b91722844f5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.646157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ae61f-e7b4-45d1-954b-7b91722844f5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.646434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ae61f-e7b4-45d1-954b-7b91722844f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.673037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-2f6ae61f-e7b4-45d1-954b-7b91722844f5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:43.674729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2f6ae61f-e7b4-45d1-954b-7b91722844f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:43.685917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:43.701146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b260657-a25c-4be5-bf22-a9526aa4b494 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.741471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.742324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.742604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.742875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.751388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.753040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.768702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 482/959] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:43.780067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:43.781071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:43.781977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 478/960] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:43.784727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ae61f-e7b4-45d1-954b-7b91722844f5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.800643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.896988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:43.905237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ae61f-e7b4-45d1-954b-7b91722844f5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.925749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34170f58-92a5-481e-9320-ecbb0d2edc69 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:43.952099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.952883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.953169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.953442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.954401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:43.954713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.956815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.963139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:43.964092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:43.964577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:43.965123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:43.966073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:43.967818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:43.969358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING keystone.server.flask.common [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 15:38:43.975753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 483/961] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:43.976660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:43.983874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.sql.core [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 15:38:43.983874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 15:38:43.983874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [parameters: {'user_id': 'a0807a924c2c4aa880b7368df31e255d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 15:38:43.983874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91957) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application conn.query(q) Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result.read() Jun 30 15:38:43.992559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 15:38:43.993462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application next(self.gen) Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application yield resource Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application next(self.gen) Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise self.value Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application session.commit() Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 15:38:43.994540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.session.flush() Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application compat.raise_( Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 15:38:43.995452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 15:38:43.996503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application conn.query(q) Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application result.read() Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 15:38:43.997380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a0807a924c2c4aa880b7368df31e255d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:43.998346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 15:38:43.999178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 15:38:44.000201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 15:38:44.000201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 15:38:44.000201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 15:38:44.000201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 15:38:44.000201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 15:38:44.000201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:44.002416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 479/962] 10.222.0.50 () {60 vars in 1131 bytes} [Thu Jun 30 15:38:43 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:44.012052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:44.026266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:44.088660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa14bd-9618-4841-be0f-5b7e749f164a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:44.127481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:44.128724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:44.129170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:44.129600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/180e36ec0b254ae1813b2892c3117d87/roles/4ab959c8c5514faa9bc905b24a221755` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:44.132245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=180e36ec0b254ae1813b2892c3117d87, role_id=4ab959c8c5514faa9bc905b24a221755)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:44.134448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:44.140808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cddd330-a4f4-4dc2-990c-c105eb8b21f5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:44.156236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 484/963] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:43 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/180e36ec0b254ae1813b2892c3117d87/roles/4ab959c8c5514faa9bc905b24a221755 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:38:44.179363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27daa3-7e56-4521-884e-84621f49879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:44.180230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27daa3-7e56-4521-884e-84621f49879e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:44.180518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27daa3-7e56-4521-884e-84621f49879e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:44.180779 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27daa3-7e56-4521-884e-84621f49879e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:44.189572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27daa3-7e56-4521-884e-84621f49879e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:44.191280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b27daa3-7e56-4521-884e-84621f49879e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-9b27daa3-7e56-4521-884e-84621f49879e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:44.207374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:44.208518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:44.210124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 480/964] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:44.215016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:44.232628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:44.374924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3f216-b436-4781-95a4-b27536a863b9 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:44.428862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383b8d9-35c1-438c-8d67-eb1112695b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:44.429952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383b8d9-35c1-438c-8d67-eb1112695b73 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:44.430438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383b8d9-35c1-438c-8d67-eb1112695b73 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:44.430909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383b8d9-35c1-438c-8d67-eb1112695b73 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:44.432473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383b8d9-35c1-438c-8d67-eb1112695b73 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:44.433005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383b8d9-35c1-438c-8d67-eb1112695b73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:44.435845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383b8d9-35c1-438c-8d67-eb1112695b73 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:44.503788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 485/965] 10.222.0.50 () {56 vars in 1101 bytes} [Thu Jun 30 15:38:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:44.877582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd8d9-0d3c-4c07-a13a-5cba8637d229 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:44.877915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd8d9-0d3c-4c07-a13a-5cba8637d229 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:44.878206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bd8d9-0d3c-4c07-a13a-5cba8637d229 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:44.882444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 481/966] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:44.887867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063c891-85ba-4862-b8e2-34ba520867dd None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:44.888156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063c891-85ba-4862-b8e2-34ba520867dd None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:44.888468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063c891-85ba-4862-b8e2-34ba520867dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:44.905459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4063c891-85ba-4862-b8e2-34ba520867dd None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:44.906654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4063c891-85ba-4862-b8e2-34ba520867dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:44.976607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 486/967] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:44.999116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bd8d9-0d3c-4c07-a13a-5cba8637d229 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:45.076110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:45.108778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bd8d9-0d3c-4c07-a13a-5cba8637d229 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:45.143346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776ab925-860b-41c2-ac46-c10d877107da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:45.143932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ab925-860b-41c2-ac46-c10d877107da None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:45.144211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ab925-860b-41c2-ac46-c10d877107da None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:45.144494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776ab925-860b-41c2-ac46-c10d877107da None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:45.145222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ab925-860b-41c2-ac46-c10d877107da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:45.146588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776ab925-860b-41c2-ac46-c10d877107da None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:45.172013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 482/968] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:44 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:45.919589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20672bc-2580-42ec-b35b-bd6d220cdf60 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:45.920286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20672bc-2580-42ec-b35b-bd6d220cdf60 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:45.920286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20672bc-2580-42ec-b35b-bd6d220cdf60 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:45.921773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 487/969] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:45.927173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1319-782e-41e2-8827-46ba533721f4 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:45.927441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1319-782e-41e2-8827-46ba533721f4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:45.927764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd1319-782e-41e2-8827-46ba533721f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:45.945805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-36cd1319-782e-41e2-8827-46ba533721f4 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:45.947424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-36cd1319-782e-41e2-8827-46ba533721f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:46.024788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 483/970] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:46.048170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20672bc-2580-42ec-b35b-bd6d220cdf60 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.051609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:46.163297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20672bc-2580-42ec-b35b-bd6d220cdf60 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.199486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd913fe-51ac-4dae-a3f7-52093876c63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.200241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd913fe-51ac-4dae-a3f7-52093876c63b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.200509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd913fe-51ac-4dae-a3f7-52093876c63b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.200779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd913fe-51ac-4dae-a3f7-52093876c63b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.201453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd913fe-51ac-4dae-a3f7-52093876c63b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:46.202831 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd913fe-51ac-4dae-a3f7-52093876c63b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-6fd913fe-51ac-4dae-a3f7-52093876c63b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:46.215452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:46.216906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:46.217881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:46.219351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 488/971] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:46.222880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd1319-782e-41e2-8827-46ba533721f4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.287151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:46.300310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438aabd-307e-429a-9150-61f8a497c345 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.300710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438aabd-307e-429a-9150-61f8a497c345 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.300883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f438aabd-307e-429a-9150-61f8a497c345 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.302476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 489/972] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:46.310342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee8951-573b-4c9d-88b2-c01e782f12e0 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.310663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee8951-573b-4c9d-88b2-c01e782f12e0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.310917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee8951-573b-4c9d-88b2-c01e782f12e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.328449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-afee8951-573b-4c9d-88b2-c01e782f12e0 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:46.329731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-afee8951-573b-4c9d-88b2-c01e782f12e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:46.338625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd1319-782e-41e2-8827-46ba533721f4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.383890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.384635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.384920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.385200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.385945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:46.386251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:46.387693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:46.393258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:46.398428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 484/973] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:46.399570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.402018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 490/974] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:46.426368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee8951-573b-4c9d-88b2-c01e782f12e0 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.508626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ca19a-8922-4636-90fc-cf7adc1749c2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.530774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee8951-573b-4c9d-88b2-c01e782f12e0 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.533405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:46.547035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.547713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.548019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.548289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.548990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:46.549306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:46.550689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:46.555834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 485/975] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:46.561517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.565761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.566587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.566866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.567130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.578480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:46.587407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:46.588646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:46.589525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 491/976] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:46.594753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.656506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:46.669179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bfef5-e9f2-49f4-9e07-bec0dc485abf None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.704351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.705067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.705312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f22444-a653-4c09-9911-36a395c62b55 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.705431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.705656 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] PATH_INFO: `/v3/endpoints` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.706459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] RBAC: Adding query filter params (interface=admin, service_id=b57abaeb507e4a148746ded344573b31, region_id=RegionOne) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:46.706777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:46.708184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:46.713581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 486/977] 10.222.0.50 () {56 vars in 1249 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/endpoints?service_id=b57abaeb507e4a148746ded344573b31&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:46.719038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.746303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.746982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.747253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.747541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.748304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:46.748615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:46.749991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:46.758787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 492/978] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:46.764090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:46.773156 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:46.840411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f7438-265e-4024-81ca-22c1939b309e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.876252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55ad44-36be-4f41-8853-3c605f5cedac None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:46.876695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.877359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.877657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.877912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.878590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:46.879961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:46.896022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 487/979] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:38:46.905892 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:46.925390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:46.926288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:46.926633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:46.926965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:46.939832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:46.941475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:46.960630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:46.961747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:46.968221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:46.968221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:46.968221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:46.968221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:46.968221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 493/980] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:46.969751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.081765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d5196-c109-41fb-84e7-b38877c875c4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.096314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:47.118228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62e90a-c35f-4534-af76-7ce6db0203e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.118959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e90a-c35f-4534-af76-7ce6db0203e2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.119234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e90a-c35f-4534-af76-7ce6db0203e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.119509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e90a-c35f-4534-af76-7ce6db0203e2 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.120258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62e90a-c35f-4534-af76-7ce6db0203e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:47.120566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62e90a-c35f-4534-af76-7ce6db0203e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.121942 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62e90a-c35f-4534-af76-7ce6db0203e2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.138990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 488/981] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:47.140695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.205572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04621-513b-47ee-89a8-31e47134beb9 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.205876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04621-513b-47ee-89a8-31e47134beb9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.206140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac04621-513b-47ee-89a8-31e47134beb9 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.207549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 489/982] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:47.212709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b05735-67b2-4fac-8a80-8cdec4d2d93a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.213003 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b05735-67b2-4fac-8a80-8cdec4d2d93a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.213279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b05735-67b2-4fac-8a80-8cdec4d2d93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.234328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-40b05735-67b2-4fac-8a80-8cdec4d2d93a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:47.235492 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-40b05735-67b2-4fac-8a80-8cdec4d2d93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:47.250137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a2beb-292e-4c54-b8fb-c76f07c518c1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.295701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.296555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.296839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.297115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.305082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.306598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.310438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 490/983] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:47.321519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:47.322505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 15:38:47.323539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:47.327343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 494/984] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:47.328610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b05735-67b2-4fac-8a80-8cdec4d2d93a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.333094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.374198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:47.435915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b05735-67b2-4fac-8a80-8cdec4d2d93a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.438928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:47.451963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb367ced-8fad-4ef6-ac71-a4dff6e3e7d6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.470695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.471490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.471864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.472256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.473169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:47.473595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.475119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.487479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.488333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.488617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.488892 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.489571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.490915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.493745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 491/985] 10.222.0.50 () {56 vars in 1149 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:47.498935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:47.505730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:47.506741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:47.508015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:47.515076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 495/986] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:47.516153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.600919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17cd93-eb20-4d60-bc00-3d1f9dc7b33a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.616568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:47.626517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233ea31-21ce-4df2-a9c7-c23625bb22ad None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.641781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.642421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.642693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.642965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.664006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.665135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.665531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.665871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.666648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] RBAC: Adding query filter params (name=compute) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:47.667027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.668499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.673318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:47.674068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 496/987] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:47.678863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.720856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.722966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.785288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940e036-3338-4b14-b3de-fe4678f7697e None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.818206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.818866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.819145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.819420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.820165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] RBAC: Adding query filter params (type=compute) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:47.820470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.821871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.826625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 497/988] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:47.827392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 492/989] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 15:38:47.832352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.833529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:47.862454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:47.893845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:47.933868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dff60-546c-45a2-8c91-b2e2ec928a92 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.945700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475b827-15a8-4365-80cb-c78a572b70ff None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:47.971359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.972191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.972469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.972737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.981016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:47.981674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:47.981954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:47.982229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:47.983015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] RBAC: Adding query filter params (interface=public, service_id=4e5b32b762054fe28f5b53569e882e53, region_id=RegionOne) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:47.983328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.983436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:47.984761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.984868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:47.990672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 498/990] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/endpoints?service_id=4e5b32b762054fe28f5b53569e882e53&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:47.996565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:48.007294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:48.008746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:48.009735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:48.009735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:48.009735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:48.009735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:48.009735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 493/991] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:48.014546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:48.091971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:48.109721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feae3c7-7ee9-40c8-94cd-5a82a27e5b2c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:48.116235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f4879-ac98-4475-8aec-665f1a4b1659 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:48.130428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:48.146901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:48.147555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:48.147834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:48.148135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:48.148814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:48.150201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:48.152828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:48.153493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:48.153826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:48.154101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:48.154821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:48.155134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:48.156572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:48.157622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 499/992] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:47 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:38:48.169142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 494/993] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:48.175007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:48.295292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae3c10-81ca-46cf-a4e8-78fb7c4c795a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:48.331189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:48.332012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:48.332329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:48.332597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:48.339553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:48.341095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:48.355668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:48.356679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:48.357603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 500/994] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:48.362908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:48.466671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375bba21-d0c7-40e8-8618-f2eb8d74ba41 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:48.504223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf106c6-2e61-4841-9b5b-a4a89b22ef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:48.504934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf106c6-2e61-4841-9b5b-a4a89b22ef09 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:48.505208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf106c6-2e61-4841-9b5b-a4a89b22ef09 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:48.505476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf106c6-2e61-4841-9b5b-a4a89b22ef09 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:48.506330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf106c6-2e61-4841-9b5b-a4a89b22ef09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:48.506631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf106c6-2e61-4841-9b5b-a4a89b22ef09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:48.508413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf106c6-2e61-4841-9b5b-a4a89b22ef09 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:48.526290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 495/995] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:48.527716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:48.540328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:48.630785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae6d7-0c11-4a4f-8acc-749fff84f209 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:48.666807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6301d9-31ab-4579-8ca5-1fe42170c2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:48.667468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6301d9-31ab-4579-8ca5-1fe42170c2c2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:48.667745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6301d9-31ab-4579-8ca5-1fe42170c2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:48.668221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6301d9-31ab-4579-8ca5-1fe42170c2c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:48.669060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6301d9-31ab-4579-8ca5-1fe42170c2c2 None admin] RBAC: Adding query filter params (role.id=4ab959c8c5514faa9bc905b24a221755, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=180e36ec0b254ae1813b2892c3117d87) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:48.669366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6301d9-31ab-4579-8ca5-1fe42170c2c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:48.670802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6301d9-31ab-4579-8ca5-1fe42170c2c2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:48.684041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 501/996] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:48 2022] GET /identity/v3/role_assignments?user.id=180e36ec0b254ae1813b2892c3117d87&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=4ab959c8c5514faa9bc905b24a221755 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:48.927646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b54591-eb19-4943-a749-0a18dcfa1bb5 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:48.928025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b54591-eb19-4943-a749-0a18dcfa1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:48.928151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b54591-eb19-4943-a749-0a18dcfa1bb5 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:48.929602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 496/997] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:48.934892 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdba49-0330-4741-b6e9-0ced5ebf6527 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:48.935144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdba49-0330-4741-b6e9-0ced5ebf6527 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:48.935454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdba49-0330-4741-b6e9-0ced5ebf6527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:48.952543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-56fdba49-0330-4741-b6e9-0ced5ebf6527 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:48.953724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-56fdba49-0330-4741-b6e9-0ced5ebf6527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:49.024251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 502/998] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:49.046979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b54591-eb19-4943-a749-0a18dcfa1bb5 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:49.085288 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:49.157642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b54591-eb19-4943-a749-0a18dcfa1bb5 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:49.195149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:49.195871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:49.196177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:49.196454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:49.197169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:49.198537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:49.210572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:49.211617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:49.212627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 497/999] 10.222.0.50 () {56 vars in 1126 bytes} [Thu Jun 30 15:38:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:49.220057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdba49-0330-4741-b6e9-0ced5ebf6527 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:49.223466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:49.332269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdba49-0330-4741-b6e9-0ced5ebf6527 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:49.367613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:49.368306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:49.368636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:49.368921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:49.369635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:49.369945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:49.371320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:49.376593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 503/1000] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:49.382296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:49.483390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:49.487702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce9ab7-fc9d-4180-a77e-efcdcc8227c3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:49.521802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c762c7b9-48bd-467e-aae0-8c81fc5b8e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:49.522461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c7b9-48bd-467e-aae0-8c81fc5b8e90 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:49.522740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c7b9-48bd-467e-aae0-8c81fc5b8e90 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:49.523010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c762c7b9-48bd-467e-aae0-8c81fc5b8e90 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:49.523742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762c7b9-48bd-467e-aae0-8c81fc5b8e90 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:49.524075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762c7b9-48bd-467e-aae0-8c81fc5b8e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:49.525555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c762c7b9-48bd-467e-aae0-8c81fc5b8e90 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:49.530564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 498/1001] 10.222.0.50 () {56 vars in 1122 bytes} [Thu Jun 30 15:38:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:49.536248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:49.605921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:49.651853 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af679531-57ef-46ae-8fa7-581116fd778d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:49.687600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8f1a9-9dd2-477d-88cb-cd02d0e52d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:49.688283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f1a9-9dd2-477d-88cb-cd02d0e52d4f None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:49.688567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f1a9-9dd2-477d-88cb-cd02d0e52d4f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:49.688834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8f1a9-9dd2-477d-88cb-cd02d0e52d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:49.689596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8f1a9-9dd2-477d-88cb-cd02d0e52d4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:49.690979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8f1a9-9dd2-477d-88cb-cd02d0e52d4f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:49.707020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 504/1002] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:50.192522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40671d-a23d-413f-99f0-1c2465cef252 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.193057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40671d-a23d-413f-99f0-1c2465cef252 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.193057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40671d-a23d-413f-99f0-1c2465cef252 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.194597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 499/1003] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:50.199883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb51d5-f12a-4d56-aaa1-2bb97c560435 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.200313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb51d5-f12a-4d56-aaa1-2bb97c560435 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.200479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdb51d5-f12a-4d56-aaa1-2bb97c560435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.218278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-afdb51d5-f12a-4d56-aaa1-2bb97c560435 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:50.219493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-afdb51d5-f12a-4d56-aaa1-2bb97c560435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:50.303199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 505/1004] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:50.326819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40671d-a23d-413f-99f0-1c2465cef252 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:50.434201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40671d-a23d-413f-99f0-1c2465cef252 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:50.469585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:50.470466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.470744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.471068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.471776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:50.473408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:50.485666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:50.492556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:50.493465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 500/1005] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:50.496391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb51d5-f12a-4d56-aaa1-2bb97c560435 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:50.557165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:50.606165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdb51d5-f12a-4d56-aaa1-2bb97c560435 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:50.641095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff0c92-acb9-4cf9-b292-df0c3707b140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:50.641771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0c92-acb9-4cf9-b292-df0c3707b140 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.642053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0c92-acb9-4cf9-b292-df0c3707b140 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.642324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0c92-acb9-4cf9-b292-df0c3707b140 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.643031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff0c92-acb9-4cf9-b292-df0c3707b140 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:50.643336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff0c92-acb9-4cf9-b292-df0c3707b140 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:50.644738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff0c92-acb9-4cf9-b292-df0c3707b140 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:50.654615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 506/1006] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:50.656103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:50.686818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:50.741459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13d747-4bf7-43a7-ae0f-efe8ddcc34ba None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.741768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13d747-4bf7-43a7-ae0f-efe8ddcc34ba None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.741949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13d747-4bf7-43a7-ae0f-efe8ddcc34ba None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.744333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 507/1007] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:50.749319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3317a8-b38c-43a6-aeeb-7497747f8ee5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.749839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3317a8-b38c-43a6-aeeb-7497747f8ee5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.749966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3317a8-b38c-43a6-aeeb-7497747f8ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.765586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea02a71-f115-4b82-ae34-c86fbbb5b0b2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:50.769571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-df3317a8-b38c-43a6-aeeb-7497747f8ee5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:50.770768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-df3317a8-b38c-43a6-aeeb-7497747f8ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:50.803782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:50.804501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.804777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.805042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.805766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:50.806135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:50.807746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:50.816721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 501/1008] 10.222.0.50 () {56 vars in 1112 bytes} [Thu Jun 30 15:38:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:50.822693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:50.848720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 508/1009] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:50.871075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3317a8-b38c-43a6-aeeb-7497747f8ee5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:50.921286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:50.924847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:50.932312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e4384-8f30-4164-bc04-1143f55c40b8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:50.964934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:50.965597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:50.965880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:50.966154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:50.966907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:50.968427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:50.981130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3317a8-b38c-43a6-aeeb-7497747f8ee5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:50.989847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 502/1010] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:51.036856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:51.037577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:51.037856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:51.038126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] PATH_INFO: `/v3/services/image` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:51.038792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:51.040246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:51.052887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:51.054033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:51.054901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:51.059792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 509/1011] 10.222.0.50 () {56 vars in 1105 bytes} [Thu Jun 30 15:38:50 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:51.061094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:51.168827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f84eb-23b7-428b-a4d5-9e64c5300b3b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:51.203529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c2d7b-ca9c-4528-a2c4-5f4a38f705b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:51.204259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c2d7b-ca9c-4528-a2c4-5f4a38f705b8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:51.204500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c2d7b-ca9c-4528-a2c4-5f4a38f705b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:51.204773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c2d7b-ca9c-4528-a2c4-5f4a38f705b8 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:51.205519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c2d7b-ca9c-4528-a2c4-5f4a38f705b8 None admin] RBAC: Adding query filter params (name=image) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:51.205823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c2d7b-ca9c-4528-a2c4-5f4a38f705b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:51.207212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c2d7b-ca9c-4528-a2c4-5f4a38f705b8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:51.213045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 503/1012] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:51.218664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:51.331477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229900db-7bca-4824-9a6f-e73fae881876 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:51.341127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:51.367569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc966d5-cb82-49fd-8b79-351541c84ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:51.368272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc966d5-cb82-49fd-8b79-351541c84ad6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:51.368579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc966d5-cb82-49fd-8b79-351541c84ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:51.368856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc966d5-cb82-49fd-8b79-351541c84ad6 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:51.369586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc966d5-cb82-49fd-8b79-351541c84ad6 None admin] RBAC: Adding query filter params (type=image) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:51.369881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc966d5-cb82-49fd-8b79-351541c84ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:51.371277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc966d5-cb82-49fd-8b79-351541c84ad6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:51.376356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 510/1013] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:51.740125 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fd3e-43fb-4687-a0d5-c0477579a508 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:51.740383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fd3e-43fb-4687-a0d5-c0477579a508 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:51.740673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97fd3e-43fb-4687-a0d5-c0477579a508 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:51.742019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 504/1014] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:51.747559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917abd0-05ba-4a09-b560-9d067c266b66 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:51.747850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917abd0-05ba-4a09-b560-9d067c266b66 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:51.748159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917abd0-05ba-4a09-b560-9d067c266b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:51.765917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4917abd0-05ba-4a09-b560-9d067c266b66 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:51.767111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4917abd0-05ba-4a09-b560-9d067c266b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:51.842376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 511/1015] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:51 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:51.865919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97fd3e-43fb-4687-a0d5-c0477579a508 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:51.880426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:51.972164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97fd3e-43fb-4687-a0d5-c0477579a508 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:52.005028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9c667-95f3-4c31-b0ac-80f5f3900f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:52.005747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9c667-95f3-4c31-b0ac-80f5f3900f58 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:52.006031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9c667-95f3-4c31-b0ac-80f5f3900f58 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:52.006308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9c667-95f3-4c31-b0ac-80f5f3900f58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:52.007824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9c667-95f3-4c31-b0ac-80f5f3900f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:52.009368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9c667-95f3-4c31-b0ac-80f5f3900f58 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:52.011171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 505/1016] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:52.016860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917abd0-05ba-4a09-b560-9d067c266b66 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:52.116356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917abd0-05ba-4a09-b560-9d067c266b66 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:52.117900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:52.152133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b7261-06ba-48a5-a54c-e1d024f5c96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:52.152890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b7261-06ba-48a5-a54c-e1d024f5c96f None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:52.153242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b7261-06ba-48a5-a54c-e1d024f5c96f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:52.153586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b7261-06ba-48a5-a54c-e1d024f5c96f None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:52.154385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b7261-06ba-48a5-a54c-e1d024f5c96f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:52.155899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b7261-06ba-48a5-a54c-e1d024f5c96f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:52.188033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 512/1017] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:52 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:53.146346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53eb25-cf55-4e29-a63c-bf12ebeb05cb None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.146927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53eb25-cf55-4e29-a63c-bf12ebeb05cb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.146927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53eb25-cf55-4e29-a63c-bf12ebeb05cb None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.148285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 506/1018] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:53.153966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75474b6-b69a-4a65-913c-79b188669885 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.154400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75474b6-b69a-4a65-913c-79b188669885 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.154497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75474b6-b69a-4a65-913c-79b188669885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.171781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b75474b6-b69a-4a65-913c-79b188669885 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:53.173046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b75474b6-b69a-4a65-913c-79b188669885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:53.241196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 513/1019] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:53.266495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53eb25-cf55-4e29-a63c-bf12ebeb05cb None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:53.355333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:53.379747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53eb25-cf55-4e29-a63c-bf12ebeb05cb None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:53.392083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eda097-1e6b-4eec-be03-2df9fbfb4b8e None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.392388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eda097-1e6b-4eec-be03-2df9fbfb4b8e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.392655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eda097-1e6b-4eec-be03-2df9fbfb4b8e None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.394287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 514/1020] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:53.399050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7974-afc4-4a6a-ba16-67723af1a66f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.399266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7974-afc4-4a6a-ba16-67723af1a66f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.399553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7974-afc4-4a6a-ba16-67723af1a66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.415961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-754b7974-afc4-4a6a-ba16-67723af1a66f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:53.416164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:53.416868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.417146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.417254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-754b7974-afc4-4a6a-ba16-67723af1a66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:53.417410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.428483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:53.436563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:53.437512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:53.438393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 507/1021] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Jun 30 15:38:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:53.444218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:53.499939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 515/1022] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:53 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:53.523517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b7974-afc4-4a6a-ba16-67723af1a66f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:53.540695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:53.559335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe75ec4-27af-47f5-83eb-403bface1ca8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:53.593269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:53.593942 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.594226 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.594546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.595239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:53.595837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:53.597017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:53.603675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:53.612310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 508/1023] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Jun 30 15:38:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:53.613195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:53.632671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b7974-afc4-4a6a-ba16-67723af1a66f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:53.667485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:53.668114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.668396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.668666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.669377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:53.670769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:53.686006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 516/1024] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:38:53 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:38:53.722406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9f24-ddd2-43da-886a-9860ce42101f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:53.767538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:53.768388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.768667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.768939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.781576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:53.783061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:53.801525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:53.802580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:53.803552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:53.803552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:53.803552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:53.803552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:53.807858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 509/1025] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:53.809356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:53.892238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:53.919699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b82fc-fee6-42fd-bed9-00ccfb38053d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:53.954743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:53.955410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:53.955689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:53.955963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:53.956721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:53.957025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:53.958400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:53.970637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 517/1026] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:53.976133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:54.026896 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:54.079497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3f43c-9d03-4094-8e56-a987594f3b8c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:54.112677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c0a4-306e-44cf-844f-70a8c61d7de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:54.113477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c0a4-306e-44cf-844f-70a8c61d7de9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.113758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c0a4-306e-44cf-844f-70a8c61d7de9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.114027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c0a4-306e-44cf-844f-70a8c61d7de9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.121943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823c0a4-306e-44cf-844f-70a8c61d7de9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:54.123403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823c0a4-306e-44cf-844f-70a8c61d7de9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-0823c0a4-306e-44cf-844f-70a8c61d7de9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:54.137745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:54.138734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:54.139727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:54.143540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 510/1027] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:38:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:54.144776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:54.246521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5b1c5-b2be-4b97-882c-bec72e31c562 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:54.248708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8914150-fed4-4aeb-b183-0e48ccfa5c08 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.249102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8914150-fed4-4aeb-b183-0e48ccfa5c08 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.249267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8914150-fed4-4aeb-b183-0e48ccfa5c08 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.250660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 511/1028] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:54.255591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1b581-f8ba-479e-8c27-0bc94fadbfc6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.255876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1b581-f8ba-479e-8c27-0bc94fadbfc6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.256185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1b581-f8ba-479e-8c27-0bc94fadbfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.273637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c6a1b581-f8ba-479e-8c27-0bc94fadbfc6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:54.274883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c6a1b581-f8ba-479e-8c27-0bc94fadbfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:54.290929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:54.291615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.292026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.292320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.293265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:54.293576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:54.295011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:54.296933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:54.316064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 518/1029] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:38:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:54.322016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:54.346593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 512/1030] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:54.371297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1b581-f8ba-479e-8c27-0bc94fadbfc6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:54.414172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:54.429186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40bc6b-4bfa-423e-97a0-50f087a613b7 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:54.467140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:54.467799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.468124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.468401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.476483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1b581-f8ba-479e-8c27-0bc94fadbfc6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:54.512499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:54.513655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.514222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.514666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.516050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:54.517833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:54.520091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 513/1031] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:54.525518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:54.532062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:54.544430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:54.545902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:54.639244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e776c07-3f46-4152-9db0-58ba98174a68 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:54.647032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:54.671357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 519/1032] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 15:38:54.672835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:54.676828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:54.677472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.677752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.678025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.678738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:54.680215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:54.703413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 514/1033] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:38:54 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:54.791833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:54.802194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f8529-6159-4cb6-a744-9643462904e1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:54.842122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:54.842965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:54.843233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:54.843504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:54.861009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:54.862433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:54.881612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:54.882867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:54.884057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:54.884057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:54.884057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:54.884057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:54.884057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 520/1034] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:54.890443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:55.002289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99c5cfd-afb9-4a0d-868f-c844b94f40a1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:55.037757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:55.038532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:55.038814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:55.039088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:55.039819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:55.040249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:55.041729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:55.054319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 515/1035] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:55.059891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:55.167432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e556572-f3bf-428d-8221-87c2d3bfe184 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:55.201562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:55.202362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:55.202646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:55.202922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:55.209804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:55.211262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:55.225092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:55.226020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:55.226926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 521/1036] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:55.233875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:55.259922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:55.345892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23873417-6ce1-4ec8-9315-b536767550b0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:55.383015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0a6c4-2d1d-427d-ac6a-23c60b4eabb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:55.383677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0a6c4-2d1d-427d-ac6a-23c60b4eabb6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:55.384009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0a6c4-2d1d-427d-ac6a-23c60b4eabb6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:55.384281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0a6c4-2d1d-427d-ac6a-23c60b4eabb6 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:55.385059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0a6c4-2d1d-427d-ac6a-23c60b4eabb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:55.385362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0a6c4-2d1d-427d-ac6a-23c60b4eabb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:55.386793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0a6c4-2d1d-427d-ac6a-23c60b4eabb6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:55.403977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 516/1037] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:55.405453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:55.410145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:55.510574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598ea7e-ed1e-4df9-bc7e-224f74f9bea2 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:55.543884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b80744-e1df-4797-93e1-cdd69364f700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:55.544575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b80744-e1df-4797-93e1-cdd69364f700 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:55.544855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b80744-e1df-4797-93e1-cdd69364f700 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:55.545130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b80744-e1df-4797-93e1-cdd69364f700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:55.545962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b80744-e1df-4797-93e1-cdd69364f700 None admin] RBAC: Adding query filter params (role.id=4ab959c8c5514faa9bc905b24a221755, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=4da1612f1c1740ceb2a918f0a8dd057b) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:55.546275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b80744-e1df-4797-93e1-cdd69364f700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:55.547722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b80744-e1df-4797-93e1-cdd69364f700 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:55.553436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 522/1038] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:55 2022] GET /identity/v3/role_assignments?user.id=4da1612f1c1740ceb2a918f0a8dd057b&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=4ab959c8c5514faa9bc905b24a221755 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:55.761916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1abd15-7883-42c5-8dd7-674a23ccccaf None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:55.762314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1abd15-7883-42c5-8dd7-674a23ccccaf None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:55.762488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1abd15-7883-42c5-8dd7-674a23ccccaf None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:55.764061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 517/1039] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:55.769105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecacf1-23ae-422a-9200-11f406c1f0d3 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:55.769398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecacf1-23ae-422a-9200-11f406c1f0d3 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:55.769682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecacf1-23ae-422a-9200-11f406c1f0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:55.787427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c0ecacf1-23ae-422a-9200-11f406c1f0d3 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:55.788640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c0ecacf1-23ae-422a-9200-11f406c1f0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:55.851535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:55.858686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 523/1040] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:55.881658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1abd15-7883-42c5-8dd7-674a23ccccaf None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:55.967914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:55.989603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1abd15-7883-42c5-8dd7-674a23ccccaf None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:56.023473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:56.024229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:56.024567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:56.024844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] PATH_INFO: `/v3/services/image` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:56.025505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:56.026885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:56.038575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:56.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:56.041086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:56.044448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 518/1041] 10.222.0.50 () {56 vars in 1105 bytes} [Thu Jun 30 15:38:55 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:56.045908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecacf1-23ae-422a-9200-11f406c1f0d3 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:56.144567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecacf1-23ae-422a-9200-11f406c1f0d3 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:56.179322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:56.180111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:56.180395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:56.180670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:56.181450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] RBAC: Adding query filter params (name=image) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:56.181757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:56.183141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:56.192352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 524/1042] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:56.193860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:56.303590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9622197-3701-4ac1-bb81-2dca506fc5b3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:56.338014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:56.339033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:56.339550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:56.340055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:56.340980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] RBAC: Adding query filter params (type=image) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:56.341488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:56.343061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:56.352800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 519/1043] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:56.354124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:56.416690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:56.456509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9495fd7-953c-448e-bb74-9288398d99bf None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:56.490661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d2ad4e-0e81-4c8a-8f74-94009d819682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:56.491329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ad4e-0e81-4c8a-8f74-94009d819682 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:56.491608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ad4e-0e81-4c8a-8f74-94009d819682 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:56.491884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d2ad4e-0e81-4c8a-8f74-94009d819682 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:56.492721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2ad4e-0e81-4c8a-8f74-94009d819682 None admin] RBAC: Adding query filter params (interface=public, service_id=bfb4171634ca4a338ae12f2e88fbcfa8, region_id=RegionOne) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:56.493037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2ad4e-0e81-4c8a-8f74-94009d819682 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:56.494459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d2ad4e-0e81-4c8a-8f74-94009d819682 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:56.504384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 525/1044] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:38:56 2022] GET /identity/v3/endpoints?service_id=bfb4171634ca4a338ae12f2e88fbcfa8&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:56.506131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:56.519598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:56.610789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0dcba-3783-4aaa-805e-f1372cc1feb9 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:56.646803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df354351-f2c0-40af-be78-5f67fc8d4da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:56.647473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df354351-f2c0-40af-be78-5f67fc8d4da6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:56.647755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df354351-f2c0-40af-be78-5f67fc8d4da6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:56.648035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df354351-f2c0-40af-be78-5f67fc8d4da6 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:56.648745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df354351-f2c0-40af-be78-5f67fc8d4da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:56.650159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df354351-f2c0-40af-be78-5f67fc8d4da6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:56.657158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 520/1045] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:38:56 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:38:56.801630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1469-45f8-4edf-9495-1d8d49d61385 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:56.802036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1469-45f8-4edf-9495-1d8d49d61385 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:56.802137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1469-45f8-4edf-9495-1d8d49d61385 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:56.803551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 526/1046] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:38:56.809573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a6167-250d-4d42-a273-89a794566ba8 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:56.809855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a6167-250d-4d42-a273-89a794566ba8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:56.810127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956a6167-250d-4d42-a273-89a794566ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:56.827781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-956a6167-250d-4d42-a273-89a794566ba8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:56.829024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-956a6167-250d-4d42-a273-89a794566ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:56.905413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 521/1047] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:38:56.928839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1469-45f8-4edf-9495-1d8d49d61385 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.033773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1469-45f8-4edf-9495-1d8d49d61385 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:57.060569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:57.070499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:57.071365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.071635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.071910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.082525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:57.090273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:57.091319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:38:57.092246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:57.096058 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 527/1048] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:38:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:57.097538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a6167-250d-4d42-a273-89a794566ba8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.101751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:57.205061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956a6167-250d-4d42-a273-89a794566ba8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:57.238369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:57.239033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.239307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.239579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.240347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:57.240648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:57.242022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:57.250580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 522/1049] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:57.256217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.363747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920066e8-146a-4d0c-8c33-98061cae8815 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:57.397715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5778d92-87c7-4329-abf3-9cf2d926e511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:57.398598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778d92-87c7-4329-abf3-9cf2d926e511 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.398872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778d92-87c7-4329-abf3-9cf2d926e511 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.399144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5778d92-87c7-4329-abf3-9cf2d926e511 None admin] PATH_INFO: `/v3/users/a7cdc293e90f4414a3040321cfd95127` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.413181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778d92-87c7-4329-abf3-9cf2d926e511 None admin] RBAC: Authorizing `identity:get_user(user_id=a7cdc293e90f4414a3040321cfd95127)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:57.414632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5778d92-87c7-4329-abf3-9cf2d926e511 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:57.416702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 528/1050] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/users/a7cdc293e90f4414a3040321cfd95127 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:57.422424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.491100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:57.528223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0989f9-960e-4273-a7a9-fdda1fd05e18 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:57.541527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ad74-5d85-4b4b-8c98-64af0ce18a43 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.542042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ad74-5d85-4b4b-8c98-64af0ce18a43 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.542123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4ad74-5d85-4b4b-8c98-64af0ce18a43 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.543656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 529/1051] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:57.548744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63cf7-af6a-48cc-9e11-f1f289ab3b39 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.549004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63cf7-af6a-48cc-9e11-f1f289ab3b39 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.549285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b63cf7-af6a-48cc-9e11-f1f289ab3b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.564157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:57.565171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.565633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.565978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d0b63cf7-af6a-48cc-9e11-f1f289ab3b39 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:57.566308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.567185 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d0b63cf7-af6a-48cc-9e11-f1f289ab3b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:57.608371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:57.639482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 530/1052] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:57.640582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:57.642046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:57.661751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63cf7-af6a-48cc-9e11-f1f289ab3b39 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.706417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:57.745583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 523/1053] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:38:57.751007 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.775825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b63cf7-af6a-48cc-9e11-f1f289ab3b39 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:57.813168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:57.813992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.814266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.814533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.825092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:38:57.833178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:57.834138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:38:57.835034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:57.840103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 531/1054] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:57.840542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.864227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e029df-124e-47b1-b363-d2863fca1acd None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:57.876052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:57.901374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:57.902431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.902710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.902978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.904243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] RBAC: Authorizing `identity:get_project(project_id=d3c46b7b00ab474199a5cf3ddb43818b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:57.905603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:57.907593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 524/1055] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:57.913148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:57.944140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b45044-67fe-4b88-9e3d-abedc9869a81 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:57.947448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:57.978369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:57.979038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:57.979316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:57.979580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:57.980683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:57.980986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:57.982326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:57.990415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 532/1056] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:57.996285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.021421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc45d0-15c6-49f0-a734-8f4a8fb0b631 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:58.058487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:58.059516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.059801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.060099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.060931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b, user.id=a7cdc293e90f4414a3040321cfd95127) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:58.061233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:58.063158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:58.080026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 525/1057] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/role_assignments?user.id=a7cdc293e90f4414a3040321cfd95127&scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:58.122543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:58.130163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b3666-435c-4dad-b18c-900bf3be9fc8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:58.168774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:58.169589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.169865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.170133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.182177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:58.183806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:58.202387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:58.203408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:58.210071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:58.210071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:58.210071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:58.210071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:58.210071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 533/1058] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:58.211729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.321768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:58.326314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650eb67-d5b0-4ac5-9087-3f51523ac249 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:58.362714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:58.363448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.363720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.364053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.364796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:58.365102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:58.366482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:58.383372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 526/1059] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:58.385486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.490570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868aa42-1b15-40f5-9314-08e3174d42d6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:58.525270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d41cbd-eb9b-469b-8159-74cda98cfbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:58.525988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d41cbd-eb9b-469b-8159-74cda98cfbf0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.526269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d41cbd-eb9b-469b-8159-74cda98cfbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.526541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d41cbd-eb9b-469b-8159-74cda98cfbf0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.534207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d41cbd-eb9b-469b-8159-74cda98cfbf0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:58.535657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d41cbd-eb9b-469b-8159-74cda98cfbf0 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-45d41cbd-eb9b-469b-8159-74cda98cfbf0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:38:58.549569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:58.550978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:38:58.552415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:58.555720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 534/1060] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:38:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:58.556969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.662371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01567bec-7879-4c0d-9bbd-6ec49c4bd534 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:58.697288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c4c3b-4aae-4ac8-9104-7cbb263c2121 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.697610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c4c3b-4aae-4ac8-9104-7cbb263c2121 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.697840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797c4c3b-4aae-4ac8-9104-7cbb263c2121 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.699362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 535/1061] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:38:58.699726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:58.700578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.700861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.701135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.701967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:58.702272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:58.704079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5cdb-2c9d-47ba-9c7e-4fb02231a639 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.704191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:58.704344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5cdb-2c9d-47ba-9c7e-4fb02231a639 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.704635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a5cdb-2c9d-47ba-9c7e-4fb02231a639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.721670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b58a5cdb-2c9d-47ba-9c7e-4fb02231a639 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:38:58.722863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b58a5cdb-2c9d-47ba-9c7e-4fb02231a639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:38:58.729332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:58.729492 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 527/1062] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:38:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:58.730260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.754653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:58.799953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 536/1063] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:38:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:38:58.827431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a5cdb-2c9d-47ba-9c7e-4fb02231a639 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.843688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0df8f-6143-41cb-9fef-caf9eea4e361 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:58.878845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:58.879743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.880211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.880575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.894383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:58.895872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:38:58.916231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:58.917331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:58.918237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:38:58.918237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:38:58.918237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:38:58.918237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:58.918237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 528/1064] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:58.923137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.935595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a5cdb-2c9d-47ba-9c7e-4fb02231a639 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:58.970325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:58.971119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:58.971452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:58.971648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] PATH_INFO: `/v3/services/image` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:58.972325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:58.973998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:58.986980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:58.988187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 15:38:58.989306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:38:58.993139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 537/1065] 10.222.0.50 () {56 vars in 1105 bytes} [Thu Jun 30 15:38:58 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:38:58.993961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:58.996349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:58.996764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:59.036828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87499989-6092-4969-aae4-2359d6b98734 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:59.071692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:59.072305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:59.072581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:59.072850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:59.073562 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:59.074043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:59.075441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:59.087499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 529/1066] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:59.093248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:59.106789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f9dc4-f6d6-4782-8296-4acedd74845c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:59.148383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:59.149260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:59.149494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:59.149764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:59.150400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] RBAC: Adding query filter params (name=image) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:59.150705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:59.152128 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:59.162770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 538/1067] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:59.163822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:59.207456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fecd4f-0293-4db6-b885-5dc6b0e8a489 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:59.230260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:59.234819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:59.247576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:59.248408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:59.248689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:59.248970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:59.255633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:59.257529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:59.270347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec7f9d-7e33-4af9-9f05-3e36c7b6abc0 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:38:59.271484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:38:59.272369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:38:59.273277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:38:59.277608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 530/1068] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:38:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:38:59.278444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:59.313391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:59.314087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:59.314369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:59.314642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:59.315463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] RBAC: Adding query filter params (type=image) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:59.315886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:59.317326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:59.327247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 539/1069] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:38:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:38:59.328162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:59.388538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84bd03-0d74-4925-89e8-1cdff9e08e8c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:59.424213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:59.424871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:59.425152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:59.425427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:59.426198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:38:59.426508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:59.427947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:59.430103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c4600-7086-41d9-bd0a-059092aa77b8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:59.439681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 531/1070] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:38:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:38:59.445259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:38:59.464150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:59.471473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca043-7440-4c79-b51e-37b78df7b168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:59.472173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca043-7440-4c79-b51e-37b78df7b168 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:59.472464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca043-7440-4c79-b51e-37b78df7b168 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:59.472742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca043-7440-4c79-b51e-37b78df7b168 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:59.473489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca043-7440-4c79-b51e-37b78df7b168 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:59.474864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca043-7440-4c79-b51e-37b78df7b168 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:59.494392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:38:59.500076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 540/1071] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:38:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:38:59.552576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b464d-263f-4043-8e79-d0612431ebb8 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:38:59.598566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a821bb-3693-444b-9c0f-1632259d6032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:38:59.599382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a821bb-3693-444b-9c0f-1632259d6032 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:38:59.599664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a821bb-3693-444b-9c0f-1632259d6032 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:38:59.599926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a821bb-3693-444b-9c0f-1632259d6032 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/4da1612f1c1740ceb2a918f0a8dd057b/roles/4ab959c8c5514faa9bc905b24a221755` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:38:59.602270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a821bb-3693-444b-9c0f-1632259d6032 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=4da1612f1c1740ceb2a918f0a8dd057b, role_id=4ab959c8c5514faa9bc905b24a221755)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:38:59.603871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a821bb-3693-444b-9c0f-1632259d6032 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:38:59.628037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 532/1072] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:38:59 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/4da1612f1c1740ceb2a918f0a8dd057b/roles/4ab959c8c5514faa9bc905b24a221755 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:39:00.081008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2645f-692a-4a72-a4cf-ff7a2a1578b2 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:00.081546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2645f-692a-4a72-a4cf-ff7a2a1578b2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:00.081546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2645f-692a-4a72-a4cf-ff7a2a1578b2 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:00.082949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 541/1073] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:00.088784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a77494-59e5-4d2f-955b-505eaf7e2639 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:00.089076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a77494-59e5-4d2f-955b-505eaf7e2639 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:00.089373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a77494-59e5-4d2f-955b-505eaf7e2639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:00.106725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-32a77494-59e5-4d2f-955b-505eaf7e2639 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:00.107921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-32a77494-59e5-4d2f-955b-505eaf7e2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:00.179167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 533/1074] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:00.201714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2645f-692a-4a72-a4cf-ff7a2a1578b2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:00.320612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2645f-692a-4a72-a4cf-ff7a2a1578b2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:00.354354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:00.355076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:00.355359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:00.355661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:00.366424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:00.373982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:00.374930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:39:00.380896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 542/1075] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:00.383922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a77494-59e5-4d2f-955b-505eaf7e2639 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:00.418866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:00.487548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a77494-59e5-4d2f-955b-505eaf7e2639 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:00.521613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:00.522284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:00.522567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:00.522839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:00.523575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:00.523934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:00.525488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:00.538895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 534/1076] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:00.539737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:00.589304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:00.649967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c15dd0-b9f8-4f5c-824a-0623cc35a223 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:00.686381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:00.687275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:00.687557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:00.687832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] PATH_INFO: `/v3/users/a7cdc293e90f4414a3040321cfd95127` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:00.689084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] RBAC: Authorizing `identity:get_user(user_id=a7cdc293e90f4414a3040321cfd95127)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:00.690537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:00.692379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 543/1077] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:00 2022] GET /identity/v3/users/a7cdc293e90f4414a3040321cfd95127 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:00.698296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:00.801105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:00.808427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a088a-d4b1-4918-bf74-6bd3b5c9247c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:00.841556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37be0-cdd6-42ab-bef6-1114dfa8f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:00.842425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37be0-cdd6-42ab-bef6-1114dfa8f97a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:00.842710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37be0-cdd6-42ab-bef6-1114dfa8f97a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:00.842977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37be0-cdd6-42ab-bef6-1114dfa8f97a None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:00.844180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb37be0-cdd6-42ab-bef6-1114dfa8f97a None admin] RBAC: Authorizing `identity:get_project(project_id=d3c46b7b00ab474199a5cf3ddb43818b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:00.845619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb37be0-cdd6-42ab-bef6-1114dfa8f97a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:00.847474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 535/1078] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:00 2022] GET /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:00.853313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:00.964586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e60e82-5ffe-4aaf-8c69-0bc739b85993 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:00.979505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:01.000207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a8c0c-f147-444c-95f2-503cbda9f020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:01.001099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8c0c-f147-444c-95f2-503cbda9f020 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:01.001379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8c0c-f147-444c-95f2-503cbda9f020 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:01.001650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a8c0c-f147-444c-95f2-503cbda9f020 None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b/users/a7cdc293e90f4414a3040321cfd95127/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:01.003580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a8c0c-f147-444c-95f2-503cbda9f020 None admin] RBAC: Authorizing `identity:create_grant(project_id=d3c46b7b00ab474199a5cf3ddb43818b, user_id=a7cdc293e90f4414a3040321cfd95127, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:01.005049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a8c0c-f147-444c-95f2-503cbda9f020 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:01.019218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 544/1079] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:39:00 2022] PUT /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b/users/a7cdc293e90f4414a3040321cfd95127/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:39:01.538913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9069ed-36fc-4037-b830-a315e0a63e88 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:01.539123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9069ed-36fc-4037-b830-a315e0a63e88 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:01.539413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9069ed-36fc-4037-b830-a315e0a63e88 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:01.541008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 536/1080] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:01.546010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a861af-eb44-4263-ad3a-cb1dfbfd993a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:01.546469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a861af-eb44-4263-ad3a-cb1dfbfd993a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:01.546568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a861af-eb44-4263-ad3a-cb1dfbfd993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:01.564021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-74a861af-eb44-4263-ad3a-cb1dfbfd993a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:01.565233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-74a861af-eb44-4263-ad3a-cb1dfbfd993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:01.632743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 545/1081] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:01.645049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d5e4f-3923-471a-813d-c46a7187b7cc None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:01.645419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d5e4f-3923-471a-813d-c46a7187b7cc None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:01.645753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d5e4f-3923-471a-813d-c46a7187b7cc None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:01.647216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 537/1082] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:01.652447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7764a2-1c0f-466a-86ee-2f3ec77a5b8d None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:01.652730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7764a2-1c0f-466a-86ee-2f3ec77a5b8d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:01.653036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7764a2-1c0f-466a-86ee-2f3ec77a5b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:01.655858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d5e4f-3923-471a-813d-c46a7187b7cc None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:01.670065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3a7764a2-1c0f-466a-86ee-2f3ec77a5b8d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:01.671238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3a7764a2-1c0f-466a-86ee-2f3ec77a5b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:01.739998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 546/1083] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:01.762179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d5e4f-3923-471a-813d-c46a7187b7cc None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:01.763737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7764a2-1c0f-466a-86ee-2f3ec77a5b8d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:01.797551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:01.798064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:01.798193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:01.798681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:01.798681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:01.831496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:01.866682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7764a2-1c0f-466a-86ee-2f3ec77a5b8d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:01.871359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:01.873076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:01.901112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:01.901926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:01.902199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:01.902466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:01.914666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:01.922784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:01.923741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:39:01.925031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 547/1084] 10.222.0.50 () {56 vars in 1120 bytes} [Thu Jun 30 15:39:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:01.929848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:01.982371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 538/1085] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:01.989145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:02.026320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:02.033165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ad72d-eec9-4398-a257-e7fc4c8b1eeb None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.056905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:02.073433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.074195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.074478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.074748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:02.075493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:02.075972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:02.077410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:02.085610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 548/1086] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Jun 30 15:39:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:02.091508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:02.103714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c85d3-9b54-474b-8e1e-012226c83f68 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.140663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.141671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.141785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.141987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:02.152809 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:02.154069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:39:02.171647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:02.172826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:02.173758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:39:02.173758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:39:02.173758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:39:02.173758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:02.173758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 539/1087] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:02.178633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:02.207417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03233f-1f82-42ab-bd24-1b3fdc0ca3f8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.241969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.242765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.243121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.243373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:02.254908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:02.256530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:39:02.277668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:02.278561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:02.280587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:39:02.280587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:39:02.280587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:39:02.280587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:39:02.280587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 549/1088] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:02.284697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:02.288710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4a9cc-8fa2-42ea-b0bb-5f8619ab8aa6 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.297539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:02.306397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:02.325071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.325666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.325984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.326252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:02.326969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:02.327324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:02.329094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:02.342734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 540/1089] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:02.394537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818fa59-acd3-4db6-8954-bf88e88f10ea None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.432377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.433121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.433458 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.433730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:02.434462 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:02.434767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:02.436216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:02.450062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 550/1090] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:02.455961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:02.562771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89990c3f-03ab-427a-8bb0-c43bd83224ac None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.597833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.598564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.598905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.599122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:02.607004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:02.608507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:39:02.622694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:02.623662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:02.624673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 541/1091] 10.222.0.50 () {56 vars in 1093 bytes} [Thu Jun 30 15:39:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:02.629829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:02.696987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:02.731723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01b84f-9aed-4804-bc15-e1d07c5caba8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.765923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db146e-8b02-43b1-8997-d17980eacad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.766590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db146e-8b02-43b1-8997-d17980eacad3 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.766863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db146e-8b02-43b1-8997-d17980eacad3 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.767133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db146e-8b02-43b1-8997-d17980eacad3 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:02.767964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db146e-8b02-43b1-8997-d17980eacad3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:02.768298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db146e-8b02-43b1-8997-d17980eacad3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:02.769725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db146e-8b02-43b1-8997-d17980eacad3 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:02.787605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 551/1092] 10.222.0.50 () {56 vars in 1133 bytes} [Thu Jun 30 15:39:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:02.793441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:02.831364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:02.898058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca1cf3-ac01-45c0-aa84-226cafee628a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:02.933189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:02.933865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:02.934117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:02.934384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.005170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.006660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.040625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.042841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aa465-55a7-4111-b11d-68e8a813301a None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.043141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aa465-55a7-4111-b11d-68e8a813301a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.043401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aa465-55a7-4111-b11d-68e8a813301a None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.044822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 552/1093] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:03.049765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7911-e128-4fd2-ab39-f2308723769c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.050029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7911-e128-4fd2-ab39-f2308723769c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.050317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7911-e128-4fd2-ab39-f2308723769c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.066878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9f0a7911-e128-4fd2-ab39-f2308723769c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:03.068108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9f0a7911-e128-4fd2-ab39-f2308723769c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:03.106166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 542/1094] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:03.111875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.137271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 553/1095] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:03.160251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a7911-e128-4fd2-ab39-f2308723769c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.214521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320341c6-afe3-42c2-a295-2299d8e580e5 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.218704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.247806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.248632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.248902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.249178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.252333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.261460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.262921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.265050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a7911-e128-4fd2-ab39-f2308723769c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:39:03.280649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:03.281782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:03.282875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:39:03.282875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:39:03.282875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:39:03.282875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:03.282875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 543/1096] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:39:03.287411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.303233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.304067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.304347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.304619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.314566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:03.322117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:03.323248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:39:03.324632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 554/1097] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:03.329288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.386517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09668ce3-9654-4202-a8ed-0dd7bcefd562 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.422476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.423068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.423349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.423620 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.424425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:03.424734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.426123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.430571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e8908-b6ea-45ff-95a3-86aabd568935 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.437827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 544/1098] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:03.443723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.447668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.465964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.466676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.466953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.467229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.467977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:03.468322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.469690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.478969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 555/1099] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:03.484814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.495239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.546112 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2080211-ec2b-4ea2-82a2-70c409af1dfd None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.579132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.579922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.580229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.580507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.583443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60eaa2-ae42-46e8-9ea4-17b81777bec2 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.587258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.588759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:39:03.603516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:03.604659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 545/1100] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:03.610618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.623867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.624800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.625080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.625352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] PATH_INFO: `/v3/users/a7cdc293e90f4414a3040321cfd95127` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.626671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] RBAC: Authorizing `identity:get_user(user_id=a7cdc293e90f4414a3040321cfd95127)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.628154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.629795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 556/1101] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/users/a7cdc293e90f4414a3040321cfd95127 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:03.635902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.685745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.709187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd59a9-8bf9-4c2c-8c87-e39cbb3cc242 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.719786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.740238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e30926-b036-4a15-902a-fd2ee2de7949 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.745720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.746332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.746610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.746879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.747662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:03.747971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.749508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.761259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 546/1102] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:03.767270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.776358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.777098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.777378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.777648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.844030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.845500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.863415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56d2e4-9b9b-4843-98ee-ea0343470641 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:03.889200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:03.896854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:03.897507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:03.897789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:03.898067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:03.898902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] RBAC: Adding query filter params (role.id=4ab959c8c5514faa9bc905b24a221755, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=4da1612f1c1740ceb2a918f0a8dd057b) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:03.899235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:03.900704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:03.906673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 547/1103] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/role_assignments?user.id=4da1612f1c1740ceb2a918f0a8dd057b&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=4ab959c8c5514faa9bc905b24a221755 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:03.944132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 557/1104] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:03.950319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:03.985205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:04.073069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576c04f-b498-47b2-bcf5-7a6ce1761c15 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:04.109011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22462a6-3bbb-4085-8d0b-c0451433567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:04.109900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22462a6-3bbb-4085-8d0b-c0451433567b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:04.110181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22462a6-3bbb-4085-8d0b-c0451433567b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:04.110456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22462a6-3bbb-4085-8d0b-c0451433567b None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:04.111619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22462a6-3bbb-4085-8d0b-c0451433567b None admin] RBAC: Authorizing `identity:get_project(project_id=d3c46b7b00ab474199a5cf3ddb43818b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:04.113132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22462a6-3bbb-4085-8d0b-c0451433567b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:04.114856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 548/1105] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:03 2022] GET /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:04.121144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:04.225392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37803532-b56b-4295-80e5-962dfedc5dab None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:04.260800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3e537-7561-4fc7-a166-81c960a3778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:04.261486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e537-7561-4fc7-a166-81c960a3778c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:04.261767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e537-7561-4fc7-a166-81c960a3778c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:04.262034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e537-7561-4fc7-a166-81c960a3778c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:04.262861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3e537-7561-4fc7-a166-81c960a3778c None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b, user.id=a7cdc293e90f4414a3040321cfd95127) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:04.263176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3e537-7561-4fc7-a166-81c960a3778c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:04.264645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3e537-7561-4fc7-a166-81c960a3778c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:04.270798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 558/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:04 2022] GET /identity/v3/role_assignments?user.id=a7cdc293e90f4414a3040321cfd95127&scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:06.334269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76a2ab-6708-43e0-a8d5-7cf31db1087c None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:06.335073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76a2ab-6708-43e0-a8d5-7cf31db1087c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:06.335073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76a2ab-6708-43e0-a8d5-7cf31db1087c None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:06.337258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 549/1107] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:06.346080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b197b-b3c5-449e-aec5-24e256d71cf2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:06.346503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b197b-b3c5-449e-aec5-24e256d71cf2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:06.346689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b197b-b3c5-449e-aec5-24e256d71cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:06.366768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-527b197b-b3c5-449e-aec5-24e256d71cf2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:06.367961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-527b197b-b3c5-449e-aec5-24e256d71cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:06.370858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:06.440045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 559/1108] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:06.467357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76a2ab-6708-43e0-a8d5-7cf31db1087c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:06.569906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:06.574552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76a2ab-6708-43e0-a8d5-7cf31db1087c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:06.621133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf8e34-f414-4a53-afd1-ac1f50278b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:06.622156 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8e34-f414-4a53-afd1-ac1f50278b9a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:06.622441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8e34-f414-4a53-afd1-ac1f50278b9a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:06.622718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8e34-f414-4a53-afd1-ac1f50278b9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:06.623891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf8e34-f414-4a53-afd1-ac1f50278b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:06.625387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf8e34-f414-4a53-afd1-ac1f50278b9a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:06.627023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 550/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:39:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:06.633271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b197b-b3c5-449e-aec5-24e256d71cf2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:06.734917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b197b-b3c5-449e-aec5-24e256d71cf2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:06.774971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26223de-686a-4813-8c2c-bacc8b93a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:06.775627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26223de-686a-4813-8c2c-bacc8b93a884 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:06.775867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26223de-686a-4813-8c2c-bacc8b93a884 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:06.776194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26223de-686a-4813-8c2c-bacc8b93a884 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:06.776912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26223de-686a-4813-8c2c-bacc8b93a884 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:06.778339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26223de-686a-4813-8c2c-bacc8b93a884 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:06.788579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:06.802836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 560/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:39:06 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:39:08.864880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1048-5baf-461a-bf6b-d7e76d9b30af None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:08.865788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1048-5baf-461a-bf6b-d7e76d9b30af None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:08.865788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a1048-5baf-461a-bf6b-d7e76d9b30af None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:08.867951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 551/1111] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:08.874474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894422b-ea26-4c64-bd13-459fa10ef5ad None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:08.875172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894422b-ea26-4c64-bd13-459fa10ef5ad None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:08.875292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894422b-ea26-4c64-bd13-459fa10ef5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:08.902327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e894422b-ea26-4c64-bd13-459fa10ef5ad None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:08.903523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e894422b-ea26-4c64-bd13-459fa10ef5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:08.972754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 561/1112] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:09.004756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1048-5baf-461a-bf6b-d7e76d9b30af None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:09.130710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a1048-5baf-461a-bf6b-d7e76d9b30af None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:09.166362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:09.167378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:09.167653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:09.167921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] PATH_INFO: `/v3/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:09.169548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] RBAC: Authorizing `identity:get_role(role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:09.170918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:09.172349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 552/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:09 2022] GET /identity/v3/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 223 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:09.179018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894422b-ea26-4c64-bd13-459fa10ef5ad None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:09.318748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894422b-ea26-4c64-bd13-459fa10ef5ad None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:09.347469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:09.356865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:09.357747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:09.358027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:09.358303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] PATH_INFO: `/v3/users/9253f832210742aabd6720ed99cc2005` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:09.371478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] RBAC: Authorizing `identity:get_user(user_id=9253f832210742aabd6720ed99cc2005)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:09.372969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:09.374650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 562/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:09 2022] GET /identity/v3/users/9253f832210742aabd6720ed99cc2005 => generated 286 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:09.380788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:09.420451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:09.490013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38562a-3916-4f00-b778-dd41ba17db72 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:09.526818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:09.527729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:09.528114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:09.528487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:09.597422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:09.598874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:09.631013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:09.701449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 553/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:09.710455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:09.819882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685dd43-3133-4ff5-beba-ada17bcaa439 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:09.870645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab3374-ecc3-4999-a5f3-b475d8cb2a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:09.871540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab3374-ecc3-4999-a5f3-b475d8cb2a09 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:09.871819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab3374-ecc3-4999-a5f3-b475d8cb2a09 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:09.872130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab3374-ecc3-4999-a5f3-b475d8cb2a09 None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:09.873399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab3374-ecc3-4999-a5f3-b475d8cb2a09 None admin] RBAC: Authorizing `identity:get_project(project_id=d3c46b7b00ab474199a5cf3ddb43818b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:09.874870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab3374-ecc3-4999-a5f3-b475d8cb2a09 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:09.876609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 563/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:09 2022] GET /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:09.884221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:09.997133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a4ca3-3494-4043-962e-026fcf40c389 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:10.032180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6fdf3-2998-46cd-8829-3ead0476a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:10.032818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fdf3-2998-46cd-8829-3ead0476a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:10.033101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fdf3-2998-46cd-8829-3ead0476a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:10.033370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6fdf3-2998-46cd-8829-3ead0476a0a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:10.034196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fdf3-2998-46cd-8829-3ead0476a0a8 None admin] RBAC: Adding query filter params (role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b, user.id=9253f832210742aabd6720ed99cc2005) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:10.034503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fdf3-2998-46cd-8829-3ead0476a0a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:10.035943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a6fdf3-2998-46cd-8829-3ead0476a0a8 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:10.038905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:10.042358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 554/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:09 2022] GET /identity/v3/role_assignments?user.id=9253f832210742aabd6720ed99cc2005&scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:12.027028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ee4b-89fd-493f-8582-c21d9cf78ff0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:12.027594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ee4b-89fd-493f-8582-c21d9cf78ff0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:12.027594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0ee4b-89fd-493f-8582-c21d9cf78ff0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:12.029001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 564/1118] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:12.035404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0559a-61ae-4a46-9d43-147090f75466 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:12.036139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0559a-61ae-4a46-9d43-147090f75466 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:12.036353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0559a-61ae-4a46-9d43-147090f75466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:12.060736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c6b0559a-61ae-4a46-9d43-147090f75466 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:12.061949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c6b0559a-61ae-4a46-9d43-147090f75466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:12.130729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 555/1119] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:12.154459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ee4b-89fd-493f-8582-c21d9cf78ff0 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:12.212380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:12.264786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0ee4b-89fd-493f-8582-c21d9cf78ff0 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:12.301111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:12.301896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:12.302178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:12.302437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] PATH_INFO: `/v3/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:12.304019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] RBAC: Authorizing `identity:get_role(role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:12.305401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:12.306822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 565/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:12 2022] GET /identity/v3/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:12.315391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0559a-61ae-4a46-9d43-147090f75466 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:12.418858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0559a-61ae-4a46-9d43-147090f75466 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:12.448749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:12.455651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:12.456548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:12.456833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:12.457108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] PATH_INFO: `/v3/users/9253f832210742aabd6720ed99cc2005` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:12.458310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] RBAC: Authorizing `identity:get_user(user_id=9253f832210742aabd6720ed99cc2005)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:12.459761 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:12.461446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 556/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:12 2022] GET /identity/v3/users/9253f832210742aabd6720ed99cc2005 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:12.470697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:12.579745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628e343-318e-49db-be1a-0a46b53da246 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:12.600397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:12.615844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27e57d-bff8-4ff9-b182-60391082a1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:12.616738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27e57d-bff8-4ff9-b182-60391082a1db None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:12.617013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27e57d-bff8-4ff9-b182-60391082a1db None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:12.617270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27e57d-bff8-4ff9-b182-60391082a1db None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:12.618610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27e57d-bff8-4ff9-b182-60391082a1db None admin] RBAC: Authorizing `identity:get_project(project_id=d3c46b7b00ab474199a5cf3ddb43818b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:12.620086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27e57d-bff8-4ff9-b182-60391082a1db None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:12.621801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 566/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:12 2022] GET /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:12.631738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:12.736649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f5732-ff98-4b91-9835-56b1bb86d6ba None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:12.772895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef61ad-8a6d-410b-865b-0db436c5284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:12.773820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef61ad-8a6d-410b-865b-0db436c5284e None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:12.774097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef61ad-8a6d-410b-865b-0db436c5284e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:12.774365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef61ad-8a6d-410b-865b-0db436c5284e None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b/users/9253f832210742aabd6720ed99cc2005/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:12.776736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef61ad-8a6d-410b-865b-0db436c5284e None admin] RBAC: Authorizing `identity:create_grant(project_id=d3c46b7b00ab474199a5cf3ddb43818b, user_id=9253f832210742aabd6720ed99cc2005, role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:12.778177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef61ad-8a6d-410b-865b-0db436c5284e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:12.792655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 557/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:39:12 2022] PUT /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b/users/9253f832210742aabd6720ed99cc2005/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:39:14.760659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46848bcd-43f8-4434-8f4d-d2a40069fbf9 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:14.761229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46848bcd-43f8-4434-8f4d-d2a40069fbf9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:14.761229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46848bcd-43f8-4434-8f4d-d2a40069fbf9 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:14.762603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 567/1124] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:14.768974 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9660166-8c48-4413-9071-f58be070cc6e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:14.769657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9660166-8c48-4413-9071-f58be070cc6e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:14.769877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9660166-8c48-4413-9071-f58be070cc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:14.796722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d9660166-8c48-4413-9071-f58be070cc6e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:14.798548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d9660166-8c48-4413-9071-f58be070cc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:14.857263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:14.872370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 558/1125] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:14.896968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46848bcd-43f8-4434-8f4d-d2a40069fbf9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:15.004961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46848bcd-43f8-4434-8f4d-d2a40069fbf9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:15.040723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:15.041498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:15.041771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:15.042038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] PATH_INFO: `/v3/roles/e28adad10c3549adbea1e6f08aa14a91` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:15.043621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] RBAC: Authorizing `identity:get_role(role_id=e28adad10c3549adbea1e6f08aa14a91)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:15.045018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:15.046405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 568/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:14 2022] GET /identity/v3/roles/e28adad10c3549adbea1e6f08aa14a91 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:15.052410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9660166-8c48-4413-9071-f58be070cc6e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:15.154621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9660166-8c48-4413-9071-f58be070cc6e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:15.189524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:15.190392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:15.190662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:15.190931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] PATH_INFO: `/v3/users/9253f832210742aabd6720ed99cc2005` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:15.192177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] RBAC: Authorizing `identity:get_user(user_id=9253f832210742aabd6720ed99cc2005)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:15.193614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:15.195223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 559/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:15 2022] GET /identity/v3/users/9253f832210742aabd6720ed99cc2005 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:15.202107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:15.277753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:15.329053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e97b0c-f00d-43ba-821b-526ff7f9b0a8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:15.369675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:15.370323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:15.370600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:15.370874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:15.450698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:15.452192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:15.510509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:15.561394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 569/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:15.567651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:15.625278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:15.669804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292922c-918d-408d-934d-9926f0ce9520 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:15.704750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc093b-f5ce-4279-8419-179d28036186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:15.705599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc093b-f5ce-4279-8419-179d28036186 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:15.705878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc093b-f5ce-4279-8419-179d28036186 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:15.706149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc093b-f5ce-4279-8419-179d28036186 None admin] PATH_INFO: `/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:15.707448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc093b-f5ce-4279-8419-179d28036186 None admin] RBAC: Authorizing `identity:get_project(project_id=d3c46b7b00ab474199a5cf3ddb43818b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:15.708930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc093b-f5ce-4279-8419-179d28036186 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:15.710632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 560/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:15 2022] GET /identity/v3/projects/d3c46b7b00ab474199a5cf3ddb43818b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:15.717530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:15.839736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea7959-368f-4497-b5b7-3afb63c22b2f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:15.878538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc74a51-0af3-4efb-abde-bb0392af4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:15.879125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc74a51-0af3-4efb-abde-bb0392af4963 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:15.879410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc74a51-0af3-4efb-abde-bb0392af4963 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:15.879686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc74a51-0af3-4efb-abde-bb0392af4963 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:15.880562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc74a51-0af3-4efb-abde-bb0392af4963 None admin] RBAC: Adding query filter params (role.id=e28adad10c3549adbea1e6f08aa14a91, scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b, user.id=9253f832210742aabd6720ed99cc2005) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:15.880872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc74a51-0af3-4efb-abde-bb0392af4963 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:15.882697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc74a51-0af3-4efb-abde-bb0392af4963 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:15.888791 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 570/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:15 2022] GET /identity/v3/role_assignments?user.id=9253f832210742aabd6720ed99cc2005&scope.project.id=d3c46b7b00ab474199a5cf3ddb43818b&role.id=e28adad10c3549adbea1e6f08aa14a91 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:17.914229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418c451-5419-440a-893b-f2b2c2b7e1ce None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:17.914819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418c451-5419-440a-893b-f2b2c2b7e1ce None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:17.914819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418c451-5419-440a-893b-f2b2c2b7e1ce None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:17.916225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 561/1131] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:17.922508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c810ec-3ca4-4c1e-85f4-2dc2e69847ec None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:17.923204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c810ec-3ca4-4c1e-85f4-2dc2e69847ec None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:17.923428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c810ec-3ca4-4c1e-85f4-2dc2e69847ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:17.944995 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-68c810ec-3ca4-4c1e-85f4-2dc2e69847ec None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:17.946184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-68c810ec-3ca4-4c1e-85f4-2dc2e69847ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:17.961981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:18.017399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 571/1132] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:18.042397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418c451-5419-440a-893b-f2b2c2b7e1ce None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:18.150612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418c451-5419-440a-893b-f2b2c2b7e1ce None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:18.174183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:18.186631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563846f-ceca-47d0-a9f9-fbccf02c1d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:18.187421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563846f-ceca-47d0-a9f9-fbccf02c1d66 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:18.187705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563846f-ceca-47d0-a9f9-fbccf02c1d66 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:18.187979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563846f-ceca-47d0-a9f9-fbccf02c1d66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:18.189235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563846f-ceca-47d0-a9f9-fbccf02c1d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:18.190683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e563846f-ceca-47d0-a9f9-fbccf02c1d66 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:18.192368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 562/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:39:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:18.198447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c810ec-3ca4-4c1e-85f4-2dc2e69847ec None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:18.301566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c810ec-3ca4-4c1e-85f4-2dc2e69847ec None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:18.336739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d426f1-2fc4-4df9-b120-e82eace9e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:18.337394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d426f1-2fc4-4df9-b120-e82eace9e92a None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:18.337672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d426f1-2fc4-4df9-b120-e82eace9e92a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:18.337942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d426f1-2fc4-4df9-b120-e82eace9e92a None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:18.338671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d426f1-2fc4-4df9-b120-e82eace9e92a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:18.340130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d426f1-2fc4-4df9-b120-e82eace9e92a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:18.369134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 572/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:39:18 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:39:20.426250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b9a0-e339-4963-aa58-0ac1dc8aa6f3 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:20.426813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b9a0-e339-4963-aa58-0ac1dc8aa6f3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:20.426813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b9a0-e339-4963-aa58-0ac1dc8aa6f3 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:20.428207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 563/1135] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:20.433926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e411c-e11e-45eb-ac2a-5d6d9c30f12d None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:20.434389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e411c-e11e-45eb-ac2a-5d6d9c30f12d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:20.434489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e411c-e11e-45eb-ac2a-5d6d9c30f12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:20.440288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:20.454658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4d2e411c-e11e-45eb-ac2a-5d6d9c30f12d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:20.455840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4d2e411c-e11e-45eb-ac2a-5d6d9c30f12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:20.530672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 573/1136] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:39:20.555836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b9a0-e339-4963-aa58-0ac1dc8aa6f3 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:20.682961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b9a0-e339-4963-aa58-0ac1dc8aa6f3 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:20.717792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d87f67-0912-4732-aba1-a90e26bcf39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:20.718580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87f67-0912-4732-aba1-a90e26bcf39f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:20.718858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87f67-0912-4732-aba1-a90e26bcf39f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:20.719157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d87f67-0912-4732-aba1-a90e26bcf39f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:20.720417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d87f67-0912-4732-aba1-a90e26bcf39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:20.721881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d87f67-0912-4732-aba1-a90e26bcf39f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:20.723493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 564/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:39:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:20.731378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e411c-e11e-45eb-ac2a-5d6d9c30f12d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:20.837873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e411c-e11e-45eb-ac2a-5d6d9c30f12d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:20.867287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:20.874112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806661a-9be0-4106-8ccb-6175cbea4822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:20.875089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806661a-9be0-4106-8ccb-6175cbea4822 None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:20.875639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806661a-9be0-4106-8ccb-6175cbea4822 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:20.876163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806661a-9be0-4106-8ccb-6175cbea4822 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:20.877013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806661a-9be0-4106-8ccb-6175cbea4822 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:20.878854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806661a-9be0-4106-8ccb-6175cbea4822 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:20.902403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 574/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:39:20 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:39:22.951878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ad834-38f8-445e-aa49-be1285dab04e None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:22.952484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ad834-38f8-445e-aa49-be1285dab04e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:22.952484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ad834-38f8-445e-aa49-be1285dab04e None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:22.953854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 565/1139] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:22.960402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d25bd-5fd1-48cc-bd39-4daec8ca307b None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:22.961093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d25bd-5fd1-48cc-bd39-4daec8ca307b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:22.961332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d25bd-5fd1-48cc-bd39-4daec8ca307b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:22.980632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ff3d25bd-5fd1-48cc-bd39-4daec8ca307b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:22.981829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ff3d25bd-5fd1-48cc-bd39-4daec8ca307b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:23.051843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 575/1140] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:23.083024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ad834-38f8-445e-aa49-be1285dab04e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:23.168801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:23.214084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ad834-38f8-445e-aa49-be1285dab04e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:23.251381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:23.252189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:23.252471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:23.252744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:23.264620 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:23.272129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:23.273267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:23.274221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 566/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:23.281109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d25bd-5fd1-48cc-bd39-4daec8ca307b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:23.384904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d25bd-5fd1-48cc-bd39-4daec8ca307b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:23.408294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:23.428238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:23.428908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:23.429192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:23.429468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:23.430263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:23.430544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:23.432019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:23.440116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 576/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:23.445993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:23.554695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be6b87-2deb-4a43-96d5-553075966915 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:23.574905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:23.591691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:23.592589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:23.592867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:23.593144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] PATH_INFO: `/v3/users/4f93d3a74afe4ce09c5ff1230cf4da72` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:23.606735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=4f93d3a74afe4ce09c5ff1230cf4da72)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:23.608281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:23.609943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 567/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:23 2022] GET /identity/v3/users/4f93d3a74afe4ce09c5ff1230cf4da72 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:23.616625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:23.741643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814aeab-af68-4a5d-8138-ffe2b31d427b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:23.778443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:23.779085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:23.779362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:23.779633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:23.828563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:23.860515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:23.861983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:23.968863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 577/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:23.975615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:24.083188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb78e56-f4d6-49cb-a89a-2b27df141aa4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:24.122612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d1b66-bbfa-4f6a-b831-cf46b629ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:24.123480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1b66-bbfa-4f6a-b831-cf46b629ac59 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:24.123760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1b66-bbfa-4f6a-b831-cf46b629ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:24.124039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d1b66-bbfa-4f6a-b831-cf46b629ac59 None admin] PATH_INFO: `/v3/projects/1e7de1ef6f8840c292cde47aba5cda99` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:24.125388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d1b66-bbfa-4f6a-b831-cf46b629ac59 None admin] RBAC: Authorizing `identity:get_project(project_id=1e7de1ef6f8840c292cde47aba5cda99)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:24.126855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d1b66-bbfa-4f6a-b831-cf46b629ac59 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:24.128590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 568/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:23 2022] GET /identity/v3/projects/1e7de1ef6f8840c292cde47aba5cda99 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:24.135303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:24.222471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:24.249542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626be0a-a485-4e9e-a9a9-dd0c2036179b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:24.286352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20679e87-1df3-42e3-8cac-a57141b26d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:24.287005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20679e87-1df3-42e3-8cac-a57141b26d84 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:24.287283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20679e87-1df3-42e3-8cac-a57141b26d84 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:24.287575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20679e87-1df3-42e3-8cac-a57141b26d84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:24.288444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20679e87-1df3-42e3-8cac-a57141b26d84 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=1e7de1ef6f8840c292cde47aba5cda99, user.id=4f93d3a74afe4ce09c5ff1230cf4da72) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:24.288757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20679e87-1df3-42e3-8cac-a57141b26d84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:24.290234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20679e87-1df3-42e3-8cac-a57141b26d84 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:24.296404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 578/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:24 2022] GET /identity/v3/role_assignments?user.id=4f93d3a74afe4ce09c5ff1230cf4da72&scope.project.id=1e7de1ef6f8840c292cde47aba5cda99&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:26.277243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ae41-cdde-4714-b0c2-77fb58ec1871 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:26.278011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ae41-cdde-4714-b0c2-77fb58ec1871 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:26.278118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3ae41-cdde-4714-b0c2-77fb58ec1871 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:26.280526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 569/1147] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:26.288055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c038-5aa2-481e-a21b-6be8dfeaa28d None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:26.288765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c038-5aa2-481e-a21b-6be8dfeaa28d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:26.288952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c038-5aa2-481e-a21b-6be8dfeaa28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:26.308008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e9f7c038-5aa2-481e-a21b-6be8dfeaa28d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:26.309305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e9f7c038-5aa2-481e-a21b-6be8dfeaa28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:26.379404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 579/1148] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:26.402908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ae41-cdde-4714-b0c2-77fb58ec1871 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:26.460228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:26.518518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3ae41-cdde-4714-b0c2-77fb58ec1871 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:26.554098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:26.554930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:26.555210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:26.555482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:26.565904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:26.573521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:26.574614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:26.575592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 570/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:26.581111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c038-5aa2-481e-a21b-6be8dfeaa28d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:26.657563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:26.686990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c038-5aa2-481e-a21b-6be8dfeaa28d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:26.722531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:26.723190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:26.723467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:26.723733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:26.724509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:26.724812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:26.726487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:26.735700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 580/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:26.744248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:26.852434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169c41a-56d4-4447-b81d-75f47c80781c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:26.859328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:26.897085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:26.897930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:26.898233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:26.898507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] PATH_INFO: `/v3/users/4f93d3a74afe4ce09c5ff1230cf4da72` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:26.899861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] RBAC: Authorizing `identity:get_user(user_id=4f93d3a74afe4ce09c5ff1230cf4da72)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:26.901365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:26.903004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 571/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:26 2022] GET /identity/v3/users/4f93d3a74afe4ce09c5ff1230cf4da72 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:26.909063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:27.020092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368552fc-e93d-4900-a320-18aa9e54b8d6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:27.064836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2849ad-7c68-4d8a-a9ea-805897d55512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:27.065708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2849ad-7c68-4d8a-a9ea-805897d55512 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:27.065985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2849ad-7c68-4d8a-a9ea-805897d55512 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:27.066269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2849ad-7c68-4d8a-a9ea-805897d55512 None admin] PATH_INFO: `/v3/projects/1e7de1ef6f8840c292cde47aba5cda99` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:27.067483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2849ad-7c68-4d8a-a9ea-805897d55512 None admin] RBAC: Authorizing `identity:get_project(project_id=1e7de1ef6f8840c292cde47aba5cda99)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:27.069427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2849ad-7c68-4d8a-a9ea-805897d55512 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:27.071207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 581/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:26 2022] GET /identity/v3/projects/1e7de1ef6f8840c292cde47aba5cda99 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:27.080559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:27.201606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e6c4d-93d2-49e8-a1a4-885ede739663 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:27.248934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963f841-a4c6-492c-a7d7-6583ef4b530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:27.249825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963f841-a4c6-492c-a7d7-6583ef4b530c None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:27.250111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963f841-a4c6-492c-a7d7-6583ef4b530c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:27.250383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963f841-a4c6-492c-a7d7-6583ef4b530c None admin] PATH_INFO: `/v3/projects/1e7de1ef6f8840c292cde47aba5cda99/users/4f93d3a74afe4ce09c5ff1230cf4da72/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:27.252731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963f841-a4c6-492c-a7d7-6583ef4b530c None admin] RBAC: Authorizing `identity:create_grant(project_id=1e7de1ef6f8840c292cde47aba5cda99, user_id=4f93d3a74afe4ce09c5ff1230cf4da72, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:27.254183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963f841-a4c6-492c-a7d7-6583ef4b530c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:27.270765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 572/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:39:27 2022] PUT /identity/v3/projects/1e7de1ef6f8840c292cde47aba5cda99/users/4f93d3a74afe4ce09c5ff1230cf4da72/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:39:29.239265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09965a06-e939-4d7d-a958-4018b658f8d8 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:29.240237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09965a06-e939-4d7d-a958-4018b658f8d8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:29.240237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09965a06-e939-4d7d-a958-4018b658f8d8 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:29.242381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 582/1154] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:29.250276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569de37-065b-472d-a3a3-cdbbf28cfd41 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:29.250753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569de37-065b-472d-a3a3-cdbbf28cfd41 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:29.250848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569de37-065b-472d-a3a3-cdbbf28cfd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:29.271305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6569de37-065b-472d-a3a3-cdbbf28cfd41 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:29.273283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6569de37-065b-472d-a3a3-cdbbf28cfd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:29.308481 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:29.350596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 573/1155] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:39:29.374337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09965a06-e939-4d7d-a958-4018b658f8d8 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:29.378647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:29.486119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09965a06-e939-4d7d-a958-4018b658f8d8 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:29.525838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:29.526558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:29.526836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:29.527112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:29.539173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:29.546642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:29.547728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:39:29.548802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 583/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:29.554119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569de37-065b-472d-a3a3-cdbbf28cfd41 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:29.656378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569de37-065b-472d-a3a3-cdbbf28cfd41 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:29.691648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:29.692330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:29.692608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:29.692878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:29.693616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:29.693926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:29.695332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:29.703589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 574/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:29.709529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:29.782878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:29.821866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302f8f0-f69c-4030-b94f-d00390463b64 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:29.856558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:29.857426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:29.857700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:29.857970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] PATH_INFO: `/v3/users/4f93d3a74afe4ce09c5ff1230cf4da72` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:29.859204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] RBAC: Authorizing `identity:get_user(user_id=4f93d3a74afe4ce09c5ff1230cf4da72)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:29.860675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:29.862298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 584/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:29 2022] GET /identity/v3/users/4f93d3a74afe4ce09c5ff1230cf4da72 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:29.868404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:29.899346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:29.976283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61492322-28ab-414e-b633-eaf17e22056c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:30.011718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:30.012378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:30.012662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:30.012937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:30.092247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:30.093651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:30.120950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:30.199332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 575/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:30.208761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:30.315060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87e958-8e23-4b25-a0da-43f275a7dc18 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:30.344884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:30.350737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88115e-8f25-478a-b362-d575086e7788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:30.351600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88115e-8f25-478a-b362-d575086e7788 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:30.351879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88115e-8f25-478a-b362-d575086e7788 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:30.352188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88115e-8f25-478a-b362-d575086e7788 None admin] PATH_INFO: `/v3/projects/1e7de1ef6f8840c292cde47aba5cda99` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:30.353485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88115e-8f25-478a-b362-d575086e7788 None admin] RBAC: Authorizing `identity:get_project(project_id=1e7de1ef6f8840c292cde47aba5cda99)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:30.354914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88115e-8f25-478a-b362-d575086e7788 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:30.356615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 585/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:30 2022] GET /identity/v3/projects/1e7de1ef6f8840c292cde47aba5cda99 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:30.363050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:30.467965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da44c615-e006-4a54-9aa2-ee37c4e92b46 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:30.496464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:30.503325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65bb45-4e2d-4981-8383-3207e7819b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:30.503975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65bb45-4e2d-4981-8383-3207e7819b04 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:30.504274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65bb45-4e2d-4981-8383-3207e7819b04 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:30.504547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65bb45-4e2d-4981-8383-3207e7819b04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:30.505368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65bb45-4e2d-4981-8383-3207e7819b04 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=1e7de1ef6f8840c292cde47aba5cda99, user.id=4f93d3a74afe4ce09c5ff1230cf4da72) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:30.506006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65bb45-4e2d-4981-8383-3207e7819b04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:30.507439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65bb45-4e2d-4981-8383-3207e7819b04 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:30.513494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 576/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:30 2022] GET /identity/v3/role_assignments?user.id=4f93d3a74afe4ce09c5ff1230cf4da72&scope.project.id=1e7de1ef6f8840c292cde47aba5cda99&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:32.526489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c564b-2e8c-4491-b290-716793a019af None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:32.527336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c564b-2e8c-4491-b290-716793a019af None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:32.527336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c564b-2e8c-4491-b290-716793a019af None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:32.529490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 586/1162] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:32.537421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ab8a-1dce-4969-a933-bc35d29af640 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:32.537868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ab8a-1dce-4969-a933-bc35d29af640 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:32.537984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07ab8a-1dce-4969-a933-bc35d29af640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:32.557182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9e07ab8a-1dce-4969-a933-bc35d29af640 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:32.558361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9e07ab8a-1dce-4969-a933-bc35d29af640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:32.630600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 577/1163] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:39:32.653963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c564b-2e8c-4491-b290-716793a019af None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:32.763062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c564b-2e8c-4491-b290-716793a019af None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:32.800690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:32.801200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:32.801479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:32.801746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:32.873418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:32.876105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:32.877595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:32.979323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 587/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:32.986295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07ab8a-1dce-4969-a933-bc35d29af640 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:33.102507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07ab8a-1dce-4969-a933-bc35d29af640 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:33.128524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:33.144755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e464b8-63ad-4e20-9c1f-ea3ced658c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:33.145557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e464b8-63ad-4e20-9c1f-ea3ced658c7d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:33.145837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e464b8-63ad-4e20-9c1f-ea3ced658c7d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:33.146107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e464b8-63ad-4e20-9c1f-ea3ced658c7d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:33.153031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e464b8-63ad-4e20-9c1f-ea3ced658c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:33.154487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e464b8-63ad-4e20-9c1f-ea3ced658c7d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-a7e464b8-63ad-4e20-9c1f-ea3ced658c7d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:33.168735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:33.169929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 15:39:33.171071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:33.174830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 578/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:39:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:33.175921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:33.289697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:33.292818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387bd577-26fc-4492-841b-28d3eea26745 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:33.327639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf40272-5f44-4aca-a35b-f52082610124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:33.328314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf40272-5f44-4aca-a35b-f52082610124 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:33.328589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf40272-5f44-4aca-a35b-f52082610124 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:33.328858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf40272-5f44-4aca-a35b-f52082610124 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:33.329568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf40272-5f44-4aca-a35b-f52082610124 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:33.329869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf40272-5f44-4aca-a35b-f52082610124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:33.331234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf40272-5f44-4aca-a35b-f52082610124 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:33.338067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 588/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Thu Jun 30 15:39:33 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:35.359565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b980f-4ed4-430a-8cc6-a1a4a5519716 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:35.360523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b980f-4ed4-430a-8cc6-a1a4a5519716 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:35.360523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b980f-4ed4-430a-8cc6-a1a4a5519716 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:35.362701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 579/1167] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:35.368316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb605-6995-449c-8d44-f48b67820285 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:35.368755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb605-6995-449c-8d44-f48b67820285 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:35.368857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cb605-6995-449c-8d44-f48b67820285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:35.387586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8b5cb605-6995-449c-8d44-f48b67820285 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:35.388796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8b5cb605-6995-449c-8d44-f48b67820285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:35.464500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 589/1168] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:35.488777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b980f-4ed4-430a-8cc6-a1a4a5519716 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:35.625422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b980f-4ed4-430a-8cc6-a1a4a5519716 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:35.664767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffc786-5da2-4186-95bd-f0b99da45476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:35.665418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffc786-5da2-4186-95bd-f0b99da45476 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:35.665704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffc786-5da2-4186-95bd-f0b99da45476 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:35.665980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffc786-5da2-4186-95bd-f0b99da45476 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:35.666725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffc786-5da2-4186-95bd-f0b99da45476 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:35.668154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffc786-5da2-4186-95bd-f0b99da45476 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:35.686203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 580/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Thu Jun 30 15:39:35 2022] POST /identity/v3/domains => generated 267 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:39:37.719081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d0319-351f-48e5-b78f-9d6b57cb230c None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:37.719626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d0319-351f-48e5-b78f-9d6b57cb230c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:37.719626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d0319-351f-48e5-b78f-9d6b57cb230c None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:37.721069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 590/1170] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:37.727423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02742-b2c1-454b-a78f-2724cf011ad2 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:37.728139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02742-b2c1-454b-a78f-2724cf011ad2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:37.728439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b02742-b2c1-454b-a78f-2724cf011ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:37.750695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-13b02742-b2c1-454b-a78f-2724cf011ad2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:37.751890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-13b02742-b2c1-454b-a78f-2724cf011ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:37.787467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:37.820800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 581/1171] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:37.844538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d0319-351f-48e5-b78f-9d6b57cb230c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:37.955553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:37.970911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d0319-351f-48e5-b78f-9d6b57cb230c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:38.015308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2fcb43-c207-4f14-93f9-f3ca2a95fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:38.016204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fcb43-c207-4f14-93f9-f3ca2a95fa2e None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:38.016486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fcb43-c207-4f14-93f9-f3ca2a95fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:38.016759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2fcb43-c207-4f14-93f9-f3ca2a95fa2e None admin] PATH_INFO: `/v3/domains/c629bbbf36944ea2ab1ba49957282609` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:38.028714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fcb43-c207-4f14-93f9-f3ca2a95fa2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=c629bbbf36944ea2ab1ba49957282609)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:38.030225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2fcb43-c207-4f14-93f9-f3ca2a95fa2e None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:38.031871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 591/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Thu Jun 30 15:39:37 2022] GET /identity/v3/domains/c629bbbf36944ea2ab1ba49957282609 => generated 267 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:38.040736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b02742-b2c1-454b-a78f-2724cf011ad2 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:38.141697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b02742-b2c1-454b-a78f-2724cf011ad2 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:38.176965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540833c-4725-4363-9232-745c0bcc61ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:38.177629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540833c-4725-4363-9232-745c0bcc61ca None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:38.177916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540833c-4725-4363-9232-745c0bcc61ca None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:38.178187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540833c-4725-4363-9232-745c0bcc61ca None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:38.178895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540833c-4725-4363-9232-745c0bcc61ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:38.180396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540833c-4725-4363-9232-745c0bcc61ca None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:38.209416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 582/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:39:38 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:39:40.247813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55e18-8f52-4e21-a0a5-863f9aba9663 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:40.248979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55e18-8f52-4e21-a0a5-863f9aba9663 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:40.248979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55e18-8f52-4e21-a0a5-863f9aba9663 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:40.250973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 592/1174] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:40.259467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca6ee-3f9f-4ac7-8792-a00e65bd4691 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:40.259908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca6ee-3f9f-4ac7-8792-a00e65bd4691 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:40.260024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aca6ee-3f9f-4ac7-8792-a00e65bd4691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:40.277539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-63aca6ee-3f9f-4ac7-8792-a00e65bd4691 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:40.278718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-63aca6ee-3f9f-4ac7-8792-a00e65bd4691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:40.298201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:40.350491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 583/1175] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:39:40.382715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55e18-8f52-4e21-a0a5-863f9aba9663 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:40.491914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55e18-8f52-4e21-a0a5-863f9aba9663 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:40.526776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c75f70-3b98-4f29-81e8-1afad2ebe742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:40.527568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c75f70-3b98-4f29-81e8-1afad2ebe742 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:40.527847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c75f70-3b98-4f29-81e8-1afad2ebe742 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:40.528151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c75f70-3b98-4f29-81e8-1afad2ebe742 None admin] PATH_INFO: `/v3/domains/c629bbbf36944ea2ab1ba49957282609` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:40.529413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c75f70-3b98-4f29-81e8-1afad2ebe742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c629bbbf36944ea2ab1ba49957282609)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:40.530849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c75f70-3b98-4f29-81e8-1afad2ebe742 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:40.532516 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 593/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Thu Jun 30 15:39:40 2022] GET /identity/v3/domains/c629bbbf36944ea2ab1ba49957282609 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:40.539004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca6ee-3f9f-4ac7-8792-a00e65bd4691 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:40.648082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aca6ee-3f9f-4ac7-8792-a00e65bd4691 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:40.685927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc81b538-5eb7-4032-8f0a-eb97ac5834c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:40.686826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81b538-5eb7-4032-8f0a-eb97ac5834c6 None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:40.687111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81b538-5eb7-4032-8f0a-eb97ac5834c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:40.687378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc81b538-5eb7-4032-8f0a-eb97ac5834c6 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:40.688118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81b538-5eb7-4032-8f0a-eb97ac5834c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:40.689572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc81b538-5eb7-4032-8f0a-eb97ac5834c6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:40.714115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 584/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:39:40 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:39:42.792284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed612e-4a70-4f87-83f6-6ea0d2ba1c4e None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:42.793160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed612e-4a70-4f87-83f6-6ea0d2ba1c4e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:42.793160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed612e-4a70-4f87-83f6-6ea0d2ba1c4e None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:42.795399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 594/1178] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:42.801878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa94b4-1ec4-44b8-b0a1-420a50974c2a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:42.802318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa94b4-1ec4-44b8-b0a1-420a50974c2a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:42.802469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa94b4-1ec4-44b8-b0a1-420a50974c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:42.821643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-dffa94b4-1ec4-44b8-b0a1-420a50974c2a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:42.822845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-dffa94b4-1ec4-44b8-b0a1-420a50974c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:42.824837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:42.899396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 585/1179] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:42.935669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed612e-4a70-4f87-83f6-6ea0d2ba1c4e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:42.964896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:43.045664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed612e-4a70-4f87-83f6-6ea0d2ba1c4e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:43.079893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:43.080715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:43.080988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:43.081257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:43.094061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:43.101797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:43.102987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:39:43.104254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 595/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:43.112286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa94b4-1ec4-44b8-b0a1-420a50974c2a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:43.223335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa94b4-1ec4-44b8-b0a1-420a50974c2a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:43.263871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:43.264566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:43.264843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:43.265115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:43.265891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:43.266203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:43.267680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:43.277184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:43.283478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 586/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:43.292264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:43.395009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:43.416149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638a08f2-966d-4d17-8c8d-cfbae0dc6db9 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:43.472032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:43.472947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:43.473227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:43.473502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] PATH_INFO: `/v3/users/9fd1224f1cb34365afd413b44aa68997` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:43.490924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] RBAC: Authorizing `identity:get_user(user_id=9fd1224f1cb34365afd413b44aa68997)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:43.492667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:43.494413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 596/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:43 2022] GET /identity/v3/users/9fd1224f1cb34365afd413b44aa68997 => generated 311 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:43.504131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:43.630063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0153f3-8dbc-4d7d-b721-32514c6bef1d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:43.666409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:43.666975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:43.667251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:43.667519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:43.739052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:43.741668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:43.743513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:43.865735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 587/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:39:43.875622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:44.002379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a2898-48a3-4ec2-8309-f9ff5521889c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:44.037020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0ef35-3971-479c-9220-c5ddc9e6acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:44.037881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0ef35-3971-479c-9220-c5ddc9e6acfc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:44.038161 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0ef35-3971-479c-9220-c5ddc9e6acfc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:44.038414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0ef35-3971-479c-9220-c5ddc9e6acfc None admin] PATH_INFO: `/v3/projects/a2c87e880a2c4531a1f0c3547d147b27` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:44.039683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0ef35-3971-479c-9220-c5ddc9e6acfc None admin] RBAC: Authorizing `identity:get_project(project_id=a2c87e880a2c4531a1f0c3547d147b27)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:44.041177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0ef35-3971-479c-9220-c5ddc9e6acfc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:44.042923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 597/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:43 2022] GET /identity/v3/projects/a2c87e880a2c4531a1f0c3547d147b27 => generated 361 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:44.049340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:44.175865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:44.181844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e25883-bef5-4f2b-9440-20bcb3206d5c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:44.224747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4287c-94f9-4713-b1b8-ce13177a99bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:44.225402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4287c-94f9-4713-b1b8-ce13177a99bf None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:44.225736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4287c-94f9-4713-b1b8-ce13177a99bf None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:44.226036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4287c-94f9-4713-b1b8-ce13177a99bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:44.226877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4287c-94f9-4713-b1b8-ce13177a99bf None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=a2c87e880a2c4531a1f0c3547d147b27, user.id=9fd1224f1cb34365afd413b44aa68997) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:44.227192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4287c-94f9-4713-b1b8-ce13177a99bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:44.228679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4287c-94f9-4713-b1b8-ce13177a99bf None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:44.234256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 588/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:44 2022] GET /identity/v3/role_assignments?user.id=9fd1224f1cb34365afd413b44aa68997&scope.project.id=a2c87e880a2c4531a1f0c3547d147b27&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:46.224134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919a102-c60f-405f-933d-53f1fbb7afdc None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:46.224805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919a102-c60f-405f-933d-53f1fbb7afdc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:46.224805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919a102-c60f-405f-933d-53f1fbb7afdc None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:46.226127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 598/1186] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:46.232681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0682-8ef4-4b26-a9bd-fe37ec25072a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:46.233468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0682-8ef4-4b26-a9bd-fe37ec25072a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:46.233646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0682-8ef4-4b26-a9bd-fe37ec25072a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:46.260328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-30fa0682-8ef4-4b26-a9bd-fe37ec25072a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:46.261519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-30fa0682-8ef4-4b26-a9bd-fe37ec25072a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:46.340231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 589/1187] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:39:46.363171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919a102-c60f-405f-933d-53f1fbb7afdc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:46.376065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:46.471864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919a102-c60f-405f-933d-53f1fbb7afdc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:46.507862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:46.508626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:46.508887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:46.509189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:46.520194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:46.527887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:46.529052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:46.530171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:39:46.532170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 599/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:46.535982 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa0682-8ef4-4b26-a9bd-fe37ec25072a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:46.638502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:46.646207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa0682-8ef4-4b26-a9bd-fe37ec25072a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:46.681286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:46.681934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:46.682216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:46.682485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:46.683220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:46.683529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:46.684933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:46.693397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 590/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:46.702228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:46.771062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:46.810329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4dcfa-9f8a-47a2-8302-1cf04b2fb033 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:46.846825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:46.847700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:46.847979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:46.848284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] PATH_INFO: `/v3/users/9fd1224f1cb34365afd413b44aa68997` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:46.849536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] RBAC: Authorizing `identity:get_user(user_id=9fd1224f1cb34365afd413b44aa68997)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:46.851041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:46.852703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 600/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:46 2022] GET /identity/v3/users/9fd1224f1cb34365afd413b44aa68997 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:39:46.862414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:46.972997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8d29ce-4b2a-4d22-8e48-eb880c29c381 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:47.007600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224ce29-1087-4de5-b4b5-86bb708a2c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:47.008492 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ce29-1087-4de5-b4b5-86bb708a2c64 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:47.008767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ce29-1087-4de5-b4b5-86bb708a2c64 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:47.009033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ce29-1087-4de5-b4b5-86bb708a2c64 None admin] PATH_INFO: `/v3/projects/a2c87e880a2c4531a1f0c3547d147b27` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:47.010312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224ce29-1087-4de5-b4b5-86bb708a2c64 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c87e880a2c4531a1f0c3547d147b27)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:47.011779 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224ce29-1087-4de5-b4b5-86bb708a2c64 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:47.013484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 591/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:46 2022] GET /identity/v3/projects/a2c87e880a2c4531a1f0c3547d147b27 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:47.020032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:47.139894 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ac246-9d4d-4ec0-a498-3d15c8dd4480 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:47.174795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d165184-bec5-4129-b1a8-dd8d34f816d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:47.175681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d165184-bec5-4129-b1a8-dd8d34f816d4 None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:47.175969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d165184-bec5-4129-b1a8-dd8d34f816d4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:47.176262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d165184-bec5-4129-b1a8-dd8d34f816d4 None admin] PATH_INFO: `/v3/projects/a2c87e880a2c4531a1f0c3547d147b27/users/9fd1224f1cb34365afd413b44aa68997/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:47.178331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d165184-bec5-4129-b1a8-dd8d34f816d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c87e880a2c4531a1f0c3547d147b27, user_id=9fd1224f1cb34365afd413b44aa68997, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:47.179843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d165184-bec5-4129-b1a8-dd8d34f816d4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:47.182917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:47.199423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 601/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:39:47 2022] PUT /identity/v3/projects/a2c87e880a2c4531a1f0c3547d147b27/users/9fd1224f1cb34365afd413b44aa68997/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:39:49.199536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7d5e-d2e3-4007-9f6d-3f87602e6bce None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:49.200455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7d5e-d2e3-4007-9f6d-3f87602e6bce None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:49.200455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec7d5e-d2e3-4007-9f6d-3f87602e6bce None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:49.202667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 592/1193] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:39:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:39:49.210577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f170b4-ee57-43e4-b455-634a2a5397f7 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:49.211030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f170b4-ee57-43e4-b455-634a2a5397f7 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:49.211137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f170b4-ee57-43e4-b455-634a2a5397f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:49.229966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a0f170b4-ee57-43e4-b455-634a2a5397f7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:39:49.231165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a0f170b4-ee57-43e4-b455-634a2a5397f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:39:49.306082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 602/1194] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:39:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 15:39:49.330717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec7d5e-d2e3-4007-9f6d-3f87602e6bce None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:49.365008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:49.464230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec7d5e-d2e3-4007-9f6d-3f87602e6bce None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:49.499716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:49.500520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:49.500775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:49.501068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:49.512405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:39:49.519823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:39:49.520949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:39:49.528019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 593/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:39:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:39:49.528777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f170b4-ee57-43e4-b455-634a2a5397f7 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:49.629759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f170b4-ee57-43e4-b455-634a2a5397f7 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:49.658558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:49.668276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:49.668943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:49.669227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:49.669502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:49.670250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:49.670569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:49.671962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:49.680245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 603/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:39:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:49.686447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:49.777858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:49.793432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474be387-137e-4dd3-8486-1ddbbb5a3296 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:49.829785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:49.830652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:49.830934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:49.831205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] PATH_INFO: `/v3/users/9fd1224f1cb34365afd413b44aa68997` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:49.832506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] RBAC: Authorizing `identity:get_user(user_id=9fd1224f1cb34365afd413b44aa68997)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:49.833969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:49.835642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 594/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Thu Jun 30 15:39:49 2022] GET /identity/v3/users/9fd1224f1cb34365afd413b44aa68997 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:49.841894 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:49.949465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6660ae-3bf0-48ea-9082-547f047a4448 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:49.983957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:49.984625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:49.984902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:49.985176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:50.041734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:50.059441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:50.060937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:50.165780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 604/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:39:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 15:39:50.175305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:50.286836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b148b297-ffbf-4b32-90ab-8a60ced26591 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:50.321002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed8992-84e4-48d7-9ddd-5bc482ad34fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:50.321795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed8992-84e4-48d7-9ddd-5bc482ad34fb None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:50.322075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed8992-84e4-48d7-9ddd-5bc482ad34fb None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:50.322347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed8992-84e4-48d7-9ddd-5bc482ad34fb None admin] PATH_INFO: `/v3/projects/a2c87e880a2c4531a1f0c3547d147b27` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:50.323664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed8992-84e4-48d7-9ddd-5bc482ad34fb None admin] RBAC: Authorizing `identity:get_project(project_id=a2c87e880a2c4531a1f0c3547d147b27)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:50.325148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed8992-84e4-48d7-9ddd-5bc482ad34fb None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:50.326851 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 595/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Thu Jun 30 15:39:50 2022] GET /identity/v3/projects/a2c87e880a2c4531a1f0c3547d147b27 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:39:50.333825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:39:50.426393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:39:50.440742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec50c85-4385-483e-8027-c074bcc884b8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:39:50.476401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0557dc-db79-4853-8a63-2a70cb09712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:39:50.477124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0557dc-db79-4853-8a63-2a70cb09712c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:39:50.477404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0557dc-db79-4853-8a63-2a70cb09712c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:39:50.477678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0557dc-db79-4853-8a63-2a70cb09712c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:39:50.478504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0557dc-db79-4853-8a63-2a70cb09712c None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=a2c87e880a2c4531a1f0c3547d147b27, user.id=9fd1224f1cb34365afd413b44aa68997) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:39:50.478814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0557dc-db79-4853-8a63-2a70cb09712c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:39:50.480270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0557dc-db79-4853-8a63-2a70cb09712c None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:39:50.486225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 605/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:39:50 2022] GET /identity/v3/role_assignments?user.id=9fd1224f1cb34365afd413b44aa68997&scope.project.id=a2c87e880a2c4531a1f0c3547d147b27&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:40:00.315812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:00.483796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=24 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:46.240676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596382ae-01be-4b8b-addb-1a3e19399f5f None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:46.241344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596382ae-01be-4b8b-addb-1a3e19399f5f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:46.241344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596382ae-01be-4b8b-addb-1a3e19399f5f None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:46.242830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 596/1201] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:40:46.249350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff1bd7-177f-4c80-97dd-d0bfc9372a81 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:46.249914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff1bd7-177f-4c80-97dd-d0bfc9372a81 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:46.249914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff1bd7-177f-4c80-97dd-d0bfc9372a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:46.273596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-90ff1bd7-177f-4c80-97dd-d0bfc9372a81 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:40:46.274843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-90ff1bd7-177f-4c80-97dd-d0bfc9372a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:40:46.366212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 606/1202] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:40:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:40:46.395482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596382ae-01be-4b8b-addb-1a3e19399f5f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:46.530351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596382ae-01be-4b8b-addb-1a3e19399f5f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:46.578430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:46.579344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:46.579660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:46.579938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:46.592229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:46.593937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:40:46.615604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:46.616838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:46.617842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:40:46.617842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:40:46.617842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:46.617842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:40:46.622243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 597/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:46.623722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff1bd7-177f-4c80-97dd-d0bfc9372a81 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:46.753735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff1bd7-177f-4c80-97dd-d0bfc9372a81 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:46.787788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:46.795668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1a12a-9b24-454f-abd1-987bddc1f7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:46.796398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a12a-9b24-454f-abd1-987bddc1f7e2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:46.796679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a12a-9b24-454f-abd1-987bddc1f7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:46.796947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1a12a-9b24-454f-abd1-987bddc1f7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:46.797733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1a12a-9b24-454f-abd1-987bddc1f7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:46.798000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1a12a-9b24-454f-abd1-987bddc1f7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:46.799427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e1a12a-9b24-454f-abd1-987bddc1f7e2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:46.819134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 607/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:46.821406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:46.883057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:46.949911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc9d8d-8273-4869-abe1-68241dda2f8c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:46.989580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce80ecc-3e4f-45be-ab4f-6aa0bd9036cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:46.990242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80ecc-3e4f-45be-ab4f-6aa0bd9036cb None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:46.990622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80ecc-3e4f-45be-ab4f-6aa0bd9036cb None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:46.990884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce80ecc-3e4f-45be-ab4f-6aa0bd9036cb None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:46.991824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce80ecc-3e4f-45be-ab4f-6aa0bd9036cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:46.993759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce80ecc-3e4f-45be-ab4f-6aa0bd9036cb None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:47.016086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 598/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Thu Jun 30 15:40:46 2022] POST /identity/v3/users => generated 251 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:40:49.382543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6f77-8562-407d-9ea0-ecbe8631da80 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:49.383219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6f77-8562-407d-9ea0-ecbe8631da80 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:49.383326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd6f77-8562-407d-9ea0-ecbe8631da80 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:49.385156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 608/1206] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:40:49.390656 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff33799-aec8-4f42-b266-d10f239617bb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:49.391121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff33799-aec8-4f42-b266-d10f239617bb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:49.391225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff33799-aec8-4f42-b266-d10f239617bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:49.414206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8ff33799-aec8-4f42-b266-d10f239617bb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:40:49.415512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8ff33799-aec8-4f42-b266-d10f239617bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:40:49.493106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 599/1207] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:40:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:40:49.517066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6f77-8562-407d-9ea0-ecbe8631da80 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:49.631823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd6f77-8562-407d-9ea0-ecbe8631da80 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:49.667392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:49.668243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:49.668562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:49.668829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:49.680581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:40:49.689415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:49.690723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:49.691759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 609/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:40:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:49.696853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff33799-aec8-4f42-b266-d10f239617bb None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:49.741049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:49.829810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff33799-aec8-4f42-b266-d10f239617bb None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:49.867084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:49.867785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:49.868094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:49.868373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:49.869153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:49.869462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:49.871015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:49.880053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 600/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:40:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:49.886046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:49.941664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:49.998780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad183-7aa6-4766-8ca3-c584bf225767 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:50.043611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:50.044543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:50.044831 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:50.045107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:50.061513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:50.063859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:40:50.084296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:50.085540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:50.091315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:40:50.091315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:40:50.091315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:50.091315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:50.091315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 610/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:40:50.091694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:50.199295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:50.202208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73413-a50d-4c6e-80aa-1a558b43f71e None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:50.237257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:50.237947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:50.238206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:50.238520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:50.239188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:50.239495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:50.240925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:50.254772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 601/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:40:50.260363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:50.376672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7698c-3b23-4cbe-bfe0-8d9c2b63aff8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:50.378297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:50.413805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:50.414720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:50.414996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:50.415225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] PATH_INFO: `/v3/users/placement` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:50.423579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:50.426979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:40:50.443246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:50.444544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:50.450553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 611/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:50.451488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:50.554674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feac9a-82f4-49e5-aa8f-1aa77b76e75a None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:50.592209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:50.592963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:50.593168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:50.593434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:50.594311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:50.594620 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:50.596104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:50.614648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 602/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Jun 30 15:40:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:50.624545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:50.759326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209378b2-20a4-432c-ac56-21f92cb22d9b None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:50.799776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:50.800738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:50.801034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:50.801292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:50.837959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:50.879117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:50.880633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:50.996857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 612/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:40:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:40:51.004148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:51.026632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:51.122031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeebc3b-967f-4e0e-ae97-53fa0b61847d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:51.161402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:51.162236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:51.162548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:51.162915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:51.175504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:51.177112 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:40:51.195860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:51.198064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:51.199259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:40:51.199259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:40:51.199259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:51.199259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:40:51.199259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 603/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:51.206829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:51.290494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:51.340720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92c8b1-69eb-4b01-bb39-0c04f383cfc6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:51.383767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:51.384527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:51.384801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:51.385079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:51.385978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:51.386290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:51.387713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:51.402124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 613/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:40:51.407928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:51.489102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:51.531104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661aa8e-a1ff-4988-bd2a-d410045208a2 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:51.578515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:51.578515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:51.578515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:51.578515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:51.587127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:51.588707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:40:51.603206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:51.604202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:40:51.605169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 604/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:40:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:51.610955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:51.718517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11864f41-a935-4e43-b2fb-ceff947e0872 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:51.741246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:51.753581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6ce8d-2fb6-4639-8e31-215fb59f7d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:51.754356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6ce8d-2fb6-4639-8e31-215fb59f7d14 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:51.754727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6ce8d-2fb6-4639-8e31-215fb59f7d14 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:51.755092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6ce8d-2fb6-4639-8e31-215fb59f7d14 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:51.756146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6ce8d-2fb6-4639-8e31-215fb59f7d14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:51.756608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6ce8d-2fb6-4639-8e31-215fb59f7d14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:51.758499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6ce8d-2fb6-4639-8e31-215fb59f7d14 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:51.776032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 614/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:40:51.781450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:51.888124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad02981-1501-45b4-9cc1-9085f17d86ee None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:51.924496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:51.932571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad1491-3589-43bb-85d2-9b0ff639261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:51.933638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1491-3589-43bb-85d2-9b0ff639261f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:51.933927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1491-3589-43bb-85d2-9b0ff639261f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:51.934245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad1491-3589-43bb-85d2-9b0ff639261f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:51.935517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad1491-3589-43bb-85d2-9b0ff639261f None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:51.935978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad1491-3589-43bb-85d2-9b0ff639261f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:51.937792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad1491-3589-43bb-85d2-9b0ff639261f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:51.948290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 605/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:40:51 2022] GET /identity/v3/role_assignments?user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:54.179185 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e3cc2-3dd0-4f1b-87ee-f0bed759f7db None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:54.179834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e3cc2-3dd0-4f1b-87ee-f0bed759f7db None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:54.179834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e3cc2-3dd0-4f1b-87ee-f0bed759f7db None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:54.181341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 615/1220] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:40:54.187222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786bed6-9eda-4ebc-b126-a8be773f5904 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:54.187698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786bed6-9eda-4ebc-b126-a8be773f5904 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:54.187892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786bed6-9eda-4ebc-b126-a8be773f5904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:54.205618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-0786bed6-9eda-4ebc-b126-a8be773f5904 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:40:54.206820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-0786bed6-9eda-4ebc-b126-a8be773f5904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:40:54.277814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 606/1221] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:40:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:40:54.303780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e3cc2-3dd0-4f1b-87ee-f0bed759f7db None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:54.415768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e3cc2-3dd0-4f1b-87ee-f0bed759f7db None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:54.452580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:54.453341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:54.453654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:54.454036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:54.465274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:40:54.473195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:54.474442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:54.475340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 616/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:40:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:54.481914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786bed6-9eda-4ebc-b126-a8be773f5904 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:54.590810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786bed6-9eda-4ebc-b126-a8be773f5904 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:54.617248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:54.629332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:54.630025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:54.630340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:54.630618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:54.631466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] RBAC: Adding query filter params (name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:54.632523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:54.633218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:54.642451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 607/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:40:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:54.649065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:54.697640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:54.762123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7108cb2-8b7b-4612-8c8a-5cd427eb633c None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:54.796174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:54.796973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:54.797256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:54.797531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:54.809034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:54.810590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:40:54.827911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:54.829103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:54.835233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:40:54.835233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:40:54.835233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:54.835233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:54.835233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 617/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:54.835557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:54.943377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c3c31-ef65-409b-b5bd-b47828d7d37f None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:54.979291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:54.979958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:54.980268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:54.980583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:54.981303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:54.981638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:54.983026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:54.995983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 608/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:55.002316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:55.103335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:55.110920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d8d0e-fa56-4525-b733-6447039bcdb8 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:55.148379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:55.149346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:55.149659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:55.149961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:55.157586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:55.159042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:40:55.173281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:55.174523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:55.175535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 618/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:55.180833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:55.231798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:55.289955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582d39e-2f2c-46c4-ac03-f0f5dbf53edb None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:55.325631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:55.326308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:55.326610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:55.326864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:55.327719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:55.328036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:55.329547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:55.347221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 609/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Jun 30 15:40:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:40:55.353163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:55.454000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32784590-fe67-483e-9515-2a2d6a50d293 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:55.490456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:55.491498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:55.491776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:55.492084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:55.503821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:55.505332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:40:55.528490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:55.529726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:55.530927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:40:55.530927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:40:55.530927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:55.530927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:55.530927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 619/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:55.536286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:55.644175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3985d-5c7d-47e5-b042-6f9af8c8c6b4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:55.646771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:55.681037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:55.681772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:55.682058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:55.682491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:55.683370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:55.683716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:55.685215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:55.698606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 610/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:55.704586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:55.714261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:55.813204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79324bf-7e1a-46b6-997f-e268b95083a4 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:55.847524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:55.848363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:55.848648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:55.848926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:55.856151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:55.857666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:40:55.872685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:55.873949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:40:55.875194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 620/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:55.880316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:55.991377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c76b1-bf45-45c0-89a0-98804a5cf462 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:56.030709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf89866-c0df-4c24-8779-45f67bbb8351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:56.031377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf89866-c0df-4c24-8779-45f67bbb8351 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:56.031659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf89866-c0df-4c24-8779-45f67bbb8351 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:56.031928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf89866-c0df-4c24-8779-45f67bbb8351 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:56.032956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf89866-c0df-4c24-8779-45f67bbb8351 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:56.033340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf89866-c0df-4c24-8779-45f67bbb8351 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:56.034952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf89866-c0df-4c24-8779-45f67bbb8351 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:56.050279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 611/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:40:56.056874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:56.135409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:56.171691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89d60c-5360-4b3e-a19f-6e72f297b154 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:56.207823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d090153d-78a0-4411-b05f-397a4832b22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:56.208759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090153d-78a0-4411-b05f-397a4832b22a None admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:56.209043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090153d-78a0-4411-b05f-397a4832b22a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:56.209326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d090153d-78a0-4411-b05f-397a4832b22a None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2e9bc6cff2d9453a80cfce2c7e66d2e7/roles/fbd9e52b6be84e29999adebc31c75d72` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:56.225903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090153d-78a0-4411-b05f-397a4832b22a None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=2e9bc6cff2d9453a80cfce2c7e66d2e7, role_id=fbd9e52b6be84e29999adebc31c75d72)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:56.227596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d090153d-78a0-4411-b05f-397a4832b22a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:56.242261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 621/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:40:56 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2e9bc6cff2d9453a80cfce2c7e66d2e7/roles/fbd9e52b6be84e29999adebc31c75d72 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:40:58.348789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b965bf-2073-413b-983a-45fb05d1d52b None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:58.348789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b965bf-2073-413b-983a-45fb05d1d52b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:58.349707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b965bf-2073-413b-983a-45fb05d1d52b None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:58.351248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 612/1233] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:40:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:40:58.356727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf3f5-d2ab-42cb-bec7-36af0839e0b1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:58.357196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf3f5-d2ab-42cb-bec7-36af0839e0b1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:58.357304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edf3f5-d2ab-42cb-bec7-36af0839e0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:58.374496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f9edf3f5-d2ab-42cb-bec7-36af0839e0b1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:40:58.376071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f9edf3f5-d2ab-42cb-bec7-36af0839e0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:40:58.449831 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 622/1234] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:40:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:40:58.474513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b965bf-2073-413b-983a-45fb05d1d52b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:58.502578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:58.583613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b965bf-2073-413b-983a-45fb05d1d52b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:58.618675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:58.619473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:58.619796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:58.620107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] PATH_INFO: `/v3/roles/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:58.631448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:40:58.639189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:58.640346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:40:58.641225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 613/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Thu Jun 30 15:40:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:58.647950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edf3f5-d2ab-42cb-bec7-36af0839e0b1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:58.694361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:58.754658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edf3f5-d2ab-42cb-bec7-36af0839e0b1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:58.796463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:58.797306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:58.797633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:58.797950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:58.798751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] RBAC: Adding query filter params (name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:58.799119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:58.800863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:58.811864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 623/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Thu Jun 30 15:40:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:58.818090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:58.909022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:58.926421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c807d91-1d19-4509-8ace-2fe33902587d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:58.960237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:58.961049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:58.961309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:58.961576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:58.972238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:58.973698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:40:58.990557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:58.991648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:58.992647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:40:58.992647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:40:58.992647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:40:58.992647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:40:58.992647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 614/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:58.998010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:59.113310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb39d59-8f30-4e61-9044-23c5e0905f78 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:59.122807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:59.151330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:59.152027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:59.152317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:59.152591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:59.153315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:59.153622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:59.154999 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:59.167789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 624/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:59.173726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:59.287048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a752ab-19a3-472f-a517-043584f4f124 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:59.323902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:59.324822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:59.325144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:59.325452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:59.328803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:59.335391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:59.337035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:40:59.351791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:40:59.353619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:40:59.354972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:40:59.357917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 615/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:40:59.359279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:59.473841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681aeaf2-4e56-4a4a-b137-0fdc7ee746a6 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:59.509228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:59.509942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:59.510247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:59.510467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:59.511309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:40:59.511616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:59.513119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:59.531001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 625/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Jun 30 15:40:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:40:59.536910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:59.643350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf5b0f-40a5-4aa2-9dd4-08754dece405 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:40:59.679655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c24365-4676-468d-a255-8d246875b838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:40:59.680349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c24365-4676-468d-a255-8d246875b838 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:40:59.680746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c24365-4676-468d-a255-8d246875b838 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:40:59.681017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c24365-4676-468d-a255-8d246875b838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:40:59.746149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:59.753577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c24365-4676-468d-a255-8d246875b838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:40:59.755060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c24365-4676-468d-a255-8d246875b838 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:40:59.858877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 616/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:40:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:40:59.865135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:40:59.893665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:40:59.970218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f1f90-a576-44e3-906e-79b21b0280c5 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:00.008236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:00.009114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:00.009416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:00.009733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:00.021050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:00.022638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:41:00.044763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:00.046118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:00.047364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:41:00.047364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:41:00.047364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:00.047364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:00.047364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 626/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:00.053024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c24365-4676-468d-a255-8d246875b838 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:00.170898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:00.178221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c24365-4676-468d-a255-8d246875b838 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:00.216709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:00.217529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:00.217813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:00.218115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:00.218990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:00.219332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:00.220757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:00.233451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 617/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:00.239418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:00.326524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:00.352139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b0122-59af-4b06-9c18-d9ebaf3dc9cf None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:00.392173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:00.393119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:00.393352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:00.393644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:00.401390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:00.402974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:41:00.419882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:00.420943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:00.421946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 627/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:00.427475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:00.534539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944278fa-8b3c-4888-bf82-4923f1467531 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:00.569632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106423b-07f8-4a4c-bb5c-da37d04feb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:00.570334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106423b-07f8-4a4c-bb5c-da37d04feb91 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:00.570619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106423b-07f8-4a4c-bb5c-da37d04feb91 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:00.570931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106423b-07f8-4a4c-bb5c-da37d04feb91 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:00.571844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106423b-07f8-4a4c-bb5c-da37d04feb91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:00.572201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106423b-07f8-4a4c-bb5c-da37d04feb91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:00.573991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106423b-07f8-4a4c-bb5c-da37d04feb91 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:00.593326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 618/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:00.594542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:00.703695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b316ef-750c-4c4e-8ab9-998b8d95d495 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:00.741906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a72de-8cf4-456a-8358-7dfba0456ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:00.742665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a72de-8cf4-456a-8358-7dfba0456ae2 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:00.742974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a72de-8cf4-456a-8358-7dfba0456ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:00.743226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a72de-8cf4-456a-8358-7dfba0456ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:00.744073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a72de-8cf4-456a-8358-7dfba0456ae2 None admin] RBAC: Adding query filter params (role.id=fbd9e52b6be84e29999adebc31c75d72, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:00.744383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a72de-8cf4-456a-8358-7dfba0456ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:00.745955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a72de-8cf4-456a-8358-7dfba0456ae2 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:00.753355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 628/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:41:00 2022] GET /identity/v3/role_assignments?user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=fbd9e52b6be84e29999adebc31c75d72 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:02.889229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f376-b754-4361-ad01-fb1b1203bd68 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:02.889827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f376-b754-4361-ad01-fb1b1203bd68 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:02.889827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f376-b754-4361-ad01-fb1b1203bd68 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:02.891632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 619/1247] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:41:02.897812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09013b-828b-408b-9c81-cbc2047de08f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:02.898286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09013b-828b-408b-9c81-cbc2047de08f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:02.898381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09013b-828b-408b-9c81-cbc2047de08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:02.904017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:02.918723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4a09013b-828b-408b-9c81-cbc2047de08f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:41:02.920011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4a09013b-828b-408b-9c81-cbc2047de08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:41:02.996476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 629/1248] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:41:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:41:03.023369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f376-b754-4361-ad01-fb1b1203bd68 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:03.044214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:03.145933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f376-b754-4361-ad01-fb1b1203bd68 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:03.183698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:03.184509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:03.184805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:03.185071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:03.196457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:41:03.207002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:03.208278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:03.209315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 620/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:03.215633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09013b-828b-408b-9c81-cbc2047de08f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:03.321445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09013b-828b-408b-9c81-cbc2047de08f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:03.354290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:03.359337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:03.360044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:03.360774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:03.361212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:03.362095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:03.362708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:03.364302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:03.373029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 630/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:03.379222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:03.463686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:03.487443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc6e1e-34aa-4194-b450-2c9391ec8150 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:03.526924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:03.527879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:03.528088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:03.528357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:03.540460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:03.542175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:41:03.560556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:03.561805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:03.563177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:41:03.563177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:41:03.563177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:03.563177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:03.563177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 621/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:03.568182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:03.677273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b41b1a-46f3-4541-bb0f-cd128f3f9408 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:03.714469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:03.715195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:03.715517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:03.715881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:03.716715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:03.717173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:03.718579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:03.733306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 631/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:03.739449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:03.861074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fd8ac-cfe9-47dc-a809-262d224d9821 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:03.893038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:03.903179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:03.904060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:03.904338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:03.904619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] PATH_INFO: `/v3/users/placement` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:03.913125 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:03.914629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:41:03.929152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:03.930419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:03.936574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 622/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:03.938789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:03.999961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:04.051492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a15c4b-9fc9-4c7e-863f-f04f2fa91204 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:04.093305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:04.093986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:04.094263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:04.094537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:04.095429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:04.095691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:04.097245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:04.115774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 632/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Jun 30 15:41:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:04.121968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:04.242679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534612e-909c-4204-a018-b4192e2aa98d None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:04.281701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:04.281701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:04.281701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:04.281701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:04.348215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:04.364827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:04.366483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:04.477536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 623/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:41:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:41:04.483734 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:04.602219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464496b6-2b07-42e3-b176-73a8050bae80 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:04.617156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:04.640948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:04.641752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:04.642040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:04.642312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:04.653659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:04.655186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:41:04.676082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:04.677246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:04.678195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:41:04.678195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:41:04.678195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:04.678195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:04.678195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 633/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:04.684098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:04.782949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:04.796208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f69f82-aef4-4d5b-ae07-950f8bc48408 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:04.832373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:04.833166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:04.833567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:04.833874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:04.834720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:04.835048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:04.836468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:04.848676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 624/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:04.855061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:04.984792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e39d37-30d4-406a-a827-171cd06c5203 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:05.020223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:05.021024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:05.021305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:05.021578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:05.029737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:05.031262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:41:05.048558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:05.049883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:05.051281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 634/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:41:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:05.055948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:05.171008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90bd50-e410-4a3d-b5dc-49122577dec0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:05.205388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8588f1-7522-4573-b0bf-9606e952d7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:05.206139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8588f1-7522-4573-b0bf-9606e952d7f6 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:05.206421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8588f1-7522-4573-b0bf-9606e952d7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:05.206741 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8588f1-7522-4573-b0bf-9606e952d7f6 None admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:05.207575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8588f1-7522-4573-b0bf-9606e952d7f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:05.207952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8588f1-7522-4573-b0bf-9606e952d7f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:05.209668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8588f1-7522-4573-b0bf-9606e952d7f6 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:05.222537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 625/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:05.228878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:05.258284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:05.350491 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b0406-20e7-4fb0-8076-ee9121a97814 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:05.389259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044122a-79c2-4616-b8b9-8816008469a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:05.389939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044122a-79c2-4616-b8b9-8816008469a6 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:05.390224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044122a-79c2-4616-b8b9-8816008469a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:05.390508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044122a-79c2-4616-b8b9-8816008469a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:05.391462 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044122a-79c2-4616-b8b9-8816008469a6 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:05.391797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044122a-79c2-4616-b8b9-8816008469a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:05.393560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044122a-79c2-4616-b8b9-8816008469a6 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:05.404026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 635/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:41:05 2022] GET /identity/v3/role_assignments?user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=5c802c0e130549e2ba423dbef846a513 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:07.419364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5236466-dee9-4dea-af8a-d8b8767d51e3 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:07.419981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5236466-dee9-4dea-af8a-d8b8767d51e3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:07.419981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5236466-dee9-4dea-af8a-d8b8767d51e3 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:07.421334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 626/1261] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:41:07.427823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5699-25ec-475f-8287-a6b96b7e36c9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:07.428644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5699-25ec-475f-8287-a6b96b7e36c9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:07.428796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5699-25ec-475f-8287-a6b96b7e36c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:07.453453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ab2a5699-25ec-475f-8287-a6b96b7e36c9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:41:07.454659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ab2a5699-25ec-475f-8287-a6b96b7e36c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:41:07.525111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 636/1262] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:41:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:41:07.548205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5236466-dee9-4dea-af8a-d8b8767d51e3 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:07.561011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:07.659621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5236466-dee9-4dea-af8a-d8b8767d51e3 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:07.694141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:07.694924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:07.695209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:07.695483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:07.707356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:41:07.714946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:07.716145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:07.717752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 627/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:07.725415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5699-25ec-475f-8287-a6b96b7e36c9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:07.746226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:07.828873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5699-25ec-475f-8287-a6b96b7e36c9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:07.864651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:07.865337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:07.865592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:07.865868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:07.866618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:07.866928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:07.868358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:07.877080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 637/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:07.883592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:07.963256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:07.998738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228d364-08bf-4620-a0a5-f6028f03a418 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:08.034188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:08.034988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:08.035243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:08.035553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:08.046937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:08.048448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:41:08.065372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:08.067327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:08.069444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:41:08.069444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:41:08.069444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:08.069444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:08.069444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 628/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:08.072795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:08.165280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:08.186793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e190f3f-dd1b-4c85-bcc3-f9c445b85a0f None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:08.226125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:08.226786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:08.227064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:08.227341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:08.228078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:08.228385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:08.229813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:08.241514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 638/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:08.251121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:08.366658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e26b50-94c1-452a-bd79-2006aa19e9f0 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:08.387342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:08.405672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:08.406466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:08.406747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:08.407034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:08.415699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:08.417104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:41:08.431352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:08.433078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:08.434353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 629/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:08.440018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:08.549396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86f643-086a-40f6-9013-a7c367a71992 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:08.597495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:08.598151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:08.598430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:08.598697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:08.599524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:08.599830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:08.601348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:08.610468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:08.619668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 639/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Jun 30 15:41:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:08.626243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:08.727694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c8d7e-e797-4261-94d5-52c5fdc613b1 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:08.762423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:08.763210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:08.763489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:08.763759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:08.774455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:08.775906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:41:08.793131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:08.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:08.795229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:41:08.795229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:41:08.795229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:08.795229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:08.795229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 630/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:08.800752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:08.913454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98490eed-e875-4e1f-8e74-2cef9386d022 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:08.950353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:08.951021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:08.951301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:08.951574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:08.952334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:08.952641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:08.954023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:08.969839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 640/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:08.977272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:09.016388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:09.105545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a47d1-5f82-40b4-80c0-0b3e81ca8e9b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:09.140899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:09.141694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:09.141979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:09.142253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:09.149221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:09.150673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:41:09.165258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:09.166267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:09.167262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 631/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:09.172630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:09.232925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:09.282752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a38be-2156-4f4f-b16a-0147b0bee8fc None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:09.321377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06a8ae-d140-4f5e-bdcb-d292dd1c6814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:09.322029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06a8ae-d140-4f5e-bdcb-d292dd1c6814 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:09.322310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06a8ae-d140-4f5e-bdcb-d292dd1c6814 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:09.322581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06a8ae-d140-4f5e-bdcb-d292dd1c6814 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:09.323363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06a8ae-d140-4f5e-bdcb-d292dd1c6814 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:09.323695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06a8ae-d140-4f5e-bdcb-d292dd1c6814 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:09.325176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06a8ae-d140-4f5e-bdcb-d292dd1c6814 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:09.337914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 641/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:09.346976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:09.446450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:09.457079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0f6e1-3885-4fbe-8c95-5f88a532cfd9 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:09.492176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7247ceb-03fa-4ab1-b636-c02f5b3e57e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:09.493058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247ceb-03fa-4ab1-b636-c02f5b3e57e0 None admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:09.493339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247ceb-03fa-4ab1-b636-c02f5b3e57e0 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:09.493611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7247ceb-03fa-4ab1-b636-c02f5b3e57e0 None admin] PATH_INFO: `/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2e9bc6cff2d9453a80cfce2c7e66d2e7/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:09.495609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247ceb-03fa-4ab1-b636-c02f5b3e57e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c25a6e43fdc045ada347b484dfdbe8ee, user_id=2e9bc6cff2d9453a80cfce2c7e66d2e7, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:09.497104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7247ceb-03fa-4ab1-b636-c02f5b3e57e0 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:09.513791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 632/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Thu Jun 30 15:41:09 2022] PUT /identity/v3/projects/c25a6e43fdc045ada347b484dfdbe8ee/users/2e9bc6cff2d9453a80cfce2c7e66d2e7/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:41:11.507968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfe9e-d1ce-49d9-922a-afe1a0542d7b None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:11.508611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfe9e-d1ce-49d9-922a-afe1a0542d7b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:11.508611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76dfe9e-d1ce-49d9-922a-afe1a0542d7b None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:11.509936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 642/1274] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:41:11.516408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575212c8-81da-47b2-87c3-f3b4fec5a6e1 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:11.517136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575212c8-81da-47b2-87c3-f3b4fec5a6e1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:11.517315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575212c8-81da-47b2-87c3-f3b4fec5a6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:11.538745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-575212c8-81da-47b2-87c3-f3b4fec5a6e1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:41:11.539866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-575212c8-81da-47b2-87c3-f3b4fec5a6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:41:11.611847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 633/1275] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:41:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:41:11.636335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dfe9e-d1ce-49d9-922a-afe1a0542d7b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:11.751246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76dfe9e-d1ce-49d9-922a-afe1a0542d7b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:11.769192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:11.787876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:11.788692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:11.789031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:11.789208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:11.799957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 15:41:11.807623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:11.808793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:11.809795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 643/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Thu Jun 30 15:41:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:11.815684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575212c8-81da-47b2-87c3-f3b4fec5a6e1 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:11.897673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:11.924097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575212c8-81da-47b2-87c3-f3b4fec5a6e1 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:11.959427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:11.960113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:11.960399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:11.960669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:11.961409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:11.961713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:11.963077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:11.970849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 634/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Thu Jun 30 15:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:11.979479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:12.094940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7575dfa-44d5-450d-b0bb-887b36a4a1cc None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:12.128780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:12.129567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:12.129847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:12.130110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:12.140701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:12.142159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:41:12.159337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:12.160397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:12.161774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:41:12.161774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:41:12.161774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:12.161774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:12.161774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 644/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 15:41:12.169072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:12.298738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2958b-daca-42fe-abc1-f689540f2ac4 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:12.334312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:12.334967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:12.335245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:12.335518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:12.336276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:12.336583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:12.337961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:12.350133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 635/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:12.356728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:12.394952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:12.470780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87e835-22f8-4ba7-8430-25563d02f9ea None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:12.513699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:12.514504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:12.514785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:12.515055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] PATH_INFO: `/v3/users/placement` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:12.522982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:12.524482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 15:41:12.538854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:12.539954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:12.541136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 645/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:12.547095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:12.552832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:12.664873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94263b2-f639-46a2-b08d-9fd5fb1b8755 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:12.711199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:12.711869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:12.712188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:12.712447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:12.713305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:12.713619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:12.715069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:12.732330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 636/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Thu Jun 30 15:41:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:12.738537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:12.833040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:12.844812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562a0e8-ceec-4398-ada4-2867e5175bba None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:12.881333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:12.881972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:12.882250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:12.882520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:12.950247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:12.951619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:13.042973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:13.057297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 646/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Thu Jun 30 15:41:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 15:41:13.063921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:13.134191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:13.173810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b120b-99c5-45a8-9dce-7fb77c8e031b None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:13.208508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:13.209234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:13.209512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:13.209780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:13.220381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:13.221853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 15:41:13.239474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:13.240546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:13.241675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 15:41:13.241675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 15:41:13.241675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 15:41:13.241675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:13.241675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 637/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:13.248719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:13.356594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b10284b-5dd4-4536-952b-019e5b8bc001 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:13.392654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:13.393305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:13.393580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:13.393851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:13.394602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:13.394967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:13.396376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:13.407654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 647/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Thu Jun 30 15:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:13.416913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:13.522346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c3eb6-52db-43b9-b421-73dbc7c57287 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:13.536561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:13.560807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:13.561607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:13.561897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:13.562172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:13.569081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:13.570567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 15:41:13.584362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:13.585422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: ERROR keystone.server.flask.application Jun 30 15:41:13.586613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 638/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Thu Jun 30 15:41:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:13.591574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:13.642931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:13.708651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6e204-e24c-43d0-b64e-4df9888ea37a None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:13.742788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6945037-6352-4728-b772-71db6863fa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:13.743435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6945037-6352-4728-b772-71db6863fa27 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:13.743710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6945037-6352-4728-b772-71db6863fa27 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:13.743983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6945037-6352-4728-b772-71db6863fa27 None admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:13.744800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6945037-6352-4728-b772-71db6863fa27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:13.745109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6945037-6352-4728-b772-71db6863fa27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:13.746526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6945037-6352-4728-b772-71db6863fa27 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:13.758935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 648/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Thu Jun 30 15:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:13.768086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:13.870320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e59eb7-4778-4280-80d4-8fa1713f7a3c None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:13.904820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e7ce3-740b-49a8-9f3d-9fbecc17c210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:13.905460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e7ce3-740b-49a8-9f3d-9fbecc17c210 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:13.905738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e7ce3-740b-49a8-9f3d-9fbecc17c210 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:13.906007 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e7ce3-740b-49a8-9f3d-9fbecc17c210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:13.906824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e7ce3-740b-49a8-9f3d-9fbecc17c210 None admin] RBAC: Adding query filter params (role.id=5c802c0e130549e2ba423dbef846a513, scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee, user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:13.907134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e7ce3-740b-49a8-9f3d-9fbecc17c210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:13.908596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e7ce3-740b-49a8-9f3d-9fbecc17c210 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:13.914360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 639/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Thu Jun 30 15:41:13 2022] GET /identity/v3/role_assignments?user.id=2e9bc6cff2d9453a80cfce2c7e66d2e7&scope.project.id=c25a6e43fdc045ada347b484dfdbe8ee&role.id=5c802c0e130549e2ba423dbef846a513 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:15.934689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca375-f57a-475e-973a-1db1ccc848d5 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:15.935597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca375-f57a-475e-973a-1db1ccc848d5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:15.935597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fca375-f57a-475e-973a-1db1ccc848d5 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:15.937855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 649/1288] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:41:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:41:15.944093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450b8cf-fefc-412d-8c9a-a603c19b48b8 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:15.944563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450b8cf-fefc-412d-8c9a-a603c19b48b8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:15.944712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450b8cf-fefc-412d-8c9a-a603c19b48b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:15.970729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8450b8cf-fefc-412d-8c9a-a603c19b48b8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:41:15.971927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8450b8cf-fefc-412d-8c9a-a603c19b48b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:41:16.002932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:16.042781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 640/1289] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:41:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:41:16.065532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fca375-f57a-475e-973a-1db1ccc848d5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:16.176820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fca375-f57a-475e-973a-1db1ccc848d5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:16.188934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:16.215330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:16.216156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:16.216432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:16.216701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:16.217345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:16.218737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:16.230739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:16.232556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:16.234440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 650/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:41:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:16.241540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450b8cf-fefc-412d-8c9a-a603c19b48b8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:16.366704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450b8cf-fefc-412d-8c9a-a603c19b48b8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:16.404177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc8a82-82a2-4348-abe6-fc5e74ab0c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:16.404848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc8a82-82a2-4348-abe6-fc5e74ab0c3c None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:16.405126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc8a82-82a2-4348-abe6-fc5e74ab0c3c None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:16.405398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc8a82-82a2-4348-abe6-fc5e74ab0c3c None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:16.406106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc8a82-82a2-4348-abe6-fc5e74ab0c3c None admin] RBAC: Adding query filter params (name=placement) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:16.406416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc8a82-82a2-4348-abe6-fc5e74ab0c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:16.407808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc8a82-82a2-4348-abe6-fc5e74ab0c3c None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:16.413093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 641/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Jun 30 15:41:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:16.419165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:16.523630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f1358-63bc-4ac1-9531-a5faa6974746 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:16.558295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f96c1a-dc1e-4c13-8eb0-0cc85433f1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:16.558944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f96c1a-dc1e-4c13-8eb0-0cc85433f1af None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:16.559247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f96c1a-dc1e-4c13-8eb0-0cc85433f1af None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:16.559552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f96c1a-dc1e-4c13-8eb0-0cc85433f1af None admin] PATH_INFO: `/v3/services` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:16.560286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f96c1a-dc1e-4c13-8eb0-0cc85433f1af None admin] RBAC: Adding query filter params (type=placement) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:16.560590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f96c1a-dc1e-4c13-8eb0-0cc85433f1af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:16.561987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f96c1a-dc1e-4c13-8eb0-0cc85433f1af None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:16.566725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 651/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Jun 30 15:41:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:18.571572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d67449-8a10-4ef1-bfb7-343dfe852489 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:18.572185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d67449-8a10-4ef1-bfb7-343dfe852489 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:18.572185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d67449-8a10-4ef1-bfb7-343dfe852489 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:18.573429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 642/1293] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:41:18.579716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce722b75-0a4a-419a-ab8b-997a5945897f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:18.580203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce722b75-0a4a-419a-ab8b-997a5945897f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:18.580304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce722b75-0a4a-419a-ab8b-997a5945897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:18.607800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ce722b75-0a4a-419a-ab8b-997a5945897f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:41:18.609011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ce722b75-0a4a-419a-ab8b-997a5945897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:41:18.647152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:18.687507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 652/1294] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:41:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:41:18.716276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d67449-8a10-4ef1-bfb7-343dfe852489 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:18.755319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:18.829621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d67449-8a10-4ef1-bfb7-343dfe852489 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:18.867634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e0441-ae9f-4ea1-b8bc-44d04dcb16fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:18.868309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e0441-ae9f-4ea1-b8bc-44d04dcb16fa None admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:18.868568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e0441-ae9f-4ea1-b8bc-44d04dcb16fa None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:18.868900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e0441-ae9f-4ea1-b8bc-44d04dcb16fa None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:18.869629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e0441-ae9f-4ea1-b8bc-44d04dcb16fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:18.871020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e0441-ae9f-4ea1-b8bc-44d04dcb16fa None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:18.891978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 643/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Thu Jun 30 15:41:18 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:41:20.929338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fc6b1-53a1-4c15-b4ad-000f09a81bd0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:20.930368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fc6b1-53a1-4c15-b4ad-000f09a81bd0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:20.930368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fc6b1-53a1-4c15-b4ad-000f09a81bd0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:20.932490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 653/1296] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:41:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:41:20.940329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaed50-6b48-449a-ab32-4e15684ae997 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:20.940795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaed50-6b48-449a-ab32-4e15684ae997 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:20.940897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaed50-6b48-449a-ab32-4e15684ae997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:20.959434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9ffaed50-6b48-449a-ab32-4e15684ae997 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:41:20.960657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9ffaed50-6b48-449a-ab32-4e15684ae997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:41:21.038099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 644/1297] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:41:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:41:21.063583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fc6b1-53a1-4c15-b4ad-000f09a81bd0 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:21.177608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fc6b1-53a1-4c15-b4ad-000f09a81bd0 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:21.213872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:21.214670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:21.214945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:21.215213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:21.215932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:21.217355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:21.229341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:21.230519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:21.231681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 654/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:41:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:21.236944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaed50-6b48-449a-ab32-4e15684ae997 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:21.262618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:21.346351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaed50-6b48-449a-ab32-4e15684ae997 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:21.380304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:21.380963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:21.381244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:21.381517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:21.382216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:21.382519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:21.383906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:21.388998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 645/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Jun 30 15:41:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:41:21.395832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:21.442124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:21.518595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcee4e-9a55-4a39-8f71-c258265026d1 None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:21.554011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341f9a6-a2c8-4d32-a021-51ffcff46c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:21.554664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341f9a6-a2c8-4d32-a021-51ffcff46c25 None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:21.554938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341f9a6-a2c8-4d32-a021-51ffcff46c25 None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:21.555211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341f9a6-a2c8-4d32-a021-51ffcff46c25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:21.556034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341f9a6-a2c8-4d32-a021-51ffcff46c25 None admin] RBAC: Adding query filter params (interface=public, service_id=ac097c67a9ba45f9a0da2973efce6d25, region_id=RegionOne) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:21.556335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341f9a6-a2c8-4d32-a021-51ffcff46c25 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:21.557702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341f9a6-a2c8-4d32-a021-51ffcff46c25 None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:21.562897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 655/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Thu Jun 30 15:41:21 2022] GET /identity/v3/endpoints?service_id=ac097c67a9ba45f9a0da2973efce6d25&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:21.568916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:21.650890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:21.669714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d150bba-a181-4f43-a354-76c9c28dcdd3 None admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:21.705063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d8c73-70b1-4a73-bee4-909b612c261e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:21.705719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8c73-70b1-4a73-bee4-909b612c261e None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:21.705999 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8c73-70b1-4a73-bee4-909b612c261e None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:21.706273 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d8c73-70b1-4a73-bee4-909b612c261e None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:21.706951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d8c73-70b1-4a73-bee4-909b612c261e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:21.708368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d8c73-70b1-4a73-bee4-909b612c261e None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:21.715721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 646/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Thu Jun 30 15:41:21 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:41:23.728543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9a38-ad12-4eaf-b89b-5752fe4df04f None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:23.729363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9a38-ad12-4eaf-b89b-5752fe4df04f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:23.729363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f9a38-ad12-4eaf-b89b-5752fe4df04f None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:23.731420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 656/1302] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:41:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:41:23.739822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edff80-f1f8-4590-9980-40b92bdd9c6e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:23.740296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edff80-f1f8-4590-9980-40b92bdd9c6e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:23.740394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edff80-f1f8-4590-9980-40b92bdd9c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:23.766734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-31edff80-f1f8-4590-9980-40b92bdd9c6e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:41:23.767930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-31edff80-f1f8-4590-9980-40b92bdd9c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:41:23.838734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 647/1303] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:41:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 15:41:23.862289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9a38-ad12-4eaf-b89b-5752fe4df04f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:23.971083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:23.978994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f9a38-ad12-4eaf-b89b-5752fe4df04f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:24.013432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:24.014126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:24.014410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:24.014675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] PATH_INFO: `/v3/services/placement` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:24.015331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:24.016737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 15:41:24.028549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 15:41:24.029774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application with Lock( Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self._enter() Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application return self.creator() Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application created_value = creator( Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: ERROR keystone.server.flask.application Jun 30 15:41:24.030874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 657/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Thu Jun 30 15:41:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 15:41:24.037909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edff80-f1f8-4590-9980-40b92bdd9c6e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:24.125053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:24.148225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edff80-f1f8-4590-9980-40b92bdd9c6e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:24.183468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba9cfef-d046-49a6-858c-943de92b3229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:24.184151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9cfef-d046-49a6-858c-943de92b3229 None admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:24.184432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9cfef-d046-49a6-858c-943de92b3229 None admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:24.184704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba9cfef-d046-49a6-858c-943de92b3229 None admin] PATH_INFO: `/v3/services` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:24.185463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9cfef-d046-49a6-858c-943de92b3229 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:41:24.185774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9cfef-d046-49a6-858c-943de92b3229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:24.187169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba9cfef-d046-49a6-858c-943de92b3229 None admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:24.192461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 648/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Thu Jun 30 15:41:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:41:24.198612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:41:24.305481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ece1d7-80bb-4e91-af41-44ce0168316d None admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:41:24.340094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cede16-16e6-4cce-8c2b-bc694b2d0e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:41:24.340752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cede16-16e6-4cce-8c2b-bc694b2d0e2f None admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:41:24.341031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cede16-16e6-4cce-8c2b-bc694b2d0e2f None admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:41:24.341311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cede16-16e6-4cce-8c2b-bc694b2d0e2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:41:24.342233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cede16-16e6-4cce-8c2b-bc694b2d0e2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:41:24.344678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cede16-16e6-4cce-8c2b-bc694b2d0e2f None admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:41:24.372334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 658/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Thu Jun 30 15:41:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:41:34.189285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=30 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:41:34.364452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:27.677639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286b788-b698-4a51-a096-627fb4bffe37 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:27.678205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286b788-b698-4a51-a096-627fb4bffe37 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:27.678205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286b788-b698-4a51-a096-627fb4bffe37 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:27.683160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 649/1307] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:27.684569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0c501-1bda-4bc5-b20c-233a28829608 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:27.684999 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0c501-1bda-4bc5-b20c-233a28829608 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:27.685105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0c501-1bda-4bc5-b20c-233a28829608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:27.776559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-72d0c501-1bda-4bc5-b20c-233a28829608 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:27.778327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-72d0c501-1bda-4bc5-b20c-233a28829608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:27.948682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 659/1308] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:42:27.983850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b00d1b-8674-4a8f-a03a-7d5056268d51 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:27.984037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b00d1b-8674-4a8f-a03a-7d5056268d51 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:27.984319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b00d1b-8674-4a8f-a03a-7d5056268d51 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:27.986056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 650/1309] 10.222.0.50 () {54 vars in 964 bytes} [Thu Jun 30 15:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:27.994767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927029-2f79-4280-aee8-d3c0b820255a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:27.995060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927029-2f79-4280-aee8-d3c0b820255a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:27.995363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73927029-2f79-4280-aee8-d3c0b820255a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:28.089191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-73927029-2f79-4280-aee8-d3c0b820255a None None] MFA Rules not processed for user `5e17d25a83fe48d99ddbd209fbea8486`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:28.090516 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-73927029-2f79-4280-aee8-d3c0b820255a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:28.114984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:28.183556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 660/1310] 10.222.0.50 () {58 vars in 1061 bytes} [Thu Jun 30 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:42:28.192791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b00d1b-8674-4a8f-a03a-7d5056268d51 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:28.194673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c5b00d1b-8674-4a8f-a03a-7d5056268d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:28.312630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b00d1b-8674-4a8f-a03a-7d5056268d51 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:28.390797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e59a4-4392-4ea2-8379-57d91a5341cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:28.391559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e59a4-4392-4ea2-8379-57d91a5341cc service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:28.391893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e59a4-4392-4ea2-8379-57d91a5341cc service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:28.392261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e59a4-4392-4ea2-8379-57d91a5341cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:28.393988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f66e59a4-4392-4ea2-8379-57d91a5341cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:28.448930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:28.505726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e59a4-4392-4ea2-8379-57d91a5341cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:28.507295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e59a4-4392-4ea2-8379-57d91a5341cc service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:28.527160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 651/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:42:30.852483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b02f-9721-4e4e-a87a-be6d7d65eb47 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:30.853022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b02f-9721-4e4e-a87a-be6d7d65eb47 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:30.853022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01b02f-9721-4e4e-a87a-be6d7d65eb47 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:30.854487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 661/1312] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:30.859807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5a740-9043-4b71-b337-0400b4fa74c0 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:30.860260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5a740-9043-4b71-b337-0400b4fa74c0 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:30.860492 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5a740-9043-4b71-b337-0400b4fa74c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:30.931654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8ed5a740-9043-4b71-b337-0400b4fa74c0 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:30.932902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8ed5a740-9043-4b71-b337-0400b4fa74c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:31.048602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 652/1313] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:31.078278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01b02f-9721-4e4e-a87a-be6d7d65eb47 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:31.080277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-cf01b02f-9721-4e4e-a87a-be6d7d65eb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:31.205443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01b02f-9721-4e4e-a87a-be6d7d65eb47 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:31.287376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:31.290378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d742a9-c2af-40bb-8691-4c4db6527ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:31.291046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d742a9-c2af-40bb-8691-4c4db6527ffd service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:31.291340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d742a9-c2af-40bb-8691-4c4db6527ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:31.291625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d742a9-c2af-40bb-8691-4c4db6527ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:31.294165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-90d742a9-c2af-40bb-8691-4c4db6527ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:31.415255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d742a9-c2af-40bb-8691-4c4db6527ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:31.416817 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d742a9-c2af-40bb-8691-4c4db6527ffd service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:31.437117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 662/1314] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:42:33.781557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530abd8d-1777-431f-a313-eb73e5784091 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:33.782101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530abd8d-1777-431f-a313-eb73e5784091 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:33.782101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530abd8d-1777-431f-a313-eb73e5784091 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:33.783400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 653/1315] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:33.789771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44601c26-07f6-4946-9d7a-078b9143dc0a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:33.790213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44601c26-07f6-4946-9d7a-078b9143dc0a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:33.790370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44601c26-07f6-4946-9d7a-078b9143dc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:33.864180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-44601c26-07f6-4946-9d7a-078b9143dc0a None None] MFA Rules not processed for user `563a6491794745e8b41b36d0b205e9df`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:33.865390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-44601c26-07f6-4946-9d7a-078b9143dc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:33.924737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:33.986946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 663/1316] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:34.029989 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530abd8d-1777-431f-a313-eb73e5784091 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:34.031541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-530abd8d-1777-431f-a313-eb73e5784091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:34.042789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:34.151798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530abd8d-1777-431f-a313-eb73e5784091 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:34.229806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e779f29-b67d-4be8-9bdf-0cdd865703c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:34.230440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e779f29-b67d-4be8-9bdf-0cdd865703c7 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:34.230721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e779f29-b67d-4be8-9bdf-0cdd865703c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:34.230986 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e779f29-b67d-4be8-9bdf-0cdd865703c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:34.232670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3e779f29-b67d-4be8-9bdf-0cdd865703c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:34.298863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:34.347686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e779f29-b67d-4be8-9bdf-0cdd865703c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:34.349207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e779f29-b67d-4be8-9bdf-0cdd865703c7 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:34.369107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 654/1317] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:42:36.932865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69e235-092b-401a-b03b-e23133e60e5d None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:36.933441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69e235-092b-401a-b03b-e23133e60e5d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:36.933441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69e235-092b-401a-b03b-e23133e60e5d None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:36.934810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 664/1318] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:36.941448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880c88-5dbb-4251-b1a2-50d22251e130 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:36.941929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880c88-5dbb-4251-b1a2-50d22251e130 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:36.942024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39880c88-5dbb-4251-b1a2-50d22251e130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:37.022086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-39880c88-5dbb-4251-b1a2-50d22251e130 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:37.023310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-39880c88-5dbb-4251-b1a2-50d22251e130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:37.129627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 655/1319] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:37.177507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69e235-092b-401a-b03b-e23133e60e5d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:37.179874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6e69e235-092b-401a-b03b-e23133e60e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:37.293067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69e235-092b-401a-b03b-e23133e60e5d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:37.376628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0479c5-1a9e-4eff-902b-98ec4ba8f367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:37.377287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0479c5-1a9e-4eff-902b-98ec4ba8f367 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:37.377565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0479c5-1a9e-4eff-902b-98ec4ba8f367 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:37.377833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0479c5-1a9e-4eff-902b-98ec4ba8f367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:37.379503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7a0479c5-1a9e-4eff-902b-98ec4ba8f367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:37.387171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:37.489950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0479c5-1a9e-4eff-902b-98ec4ba8f367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:37.491395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0479c5-1a9e-4eff-902b-98ec4ba8f367 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:37.512142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 665/1320] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:42:40.218775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b1618-d1e1-4e3a-857d-a0d80a9e3233 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:40.219352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b1618-d1e1-4e3a-857d-a0d80a9e3233 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:40.219352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b1618-d1e1-4e3a-857d-a0d80a9e3233 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:40.220861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 656/1321] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:40.226378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba7099-a5aa-4247-92bd-8e8bfc48b3b9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:40.226813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba7099-a5aa-4247-92bd-8e8bfc48b3b9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:40.227071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba7099-a5aa-4247-92bd-8e8bfc48b3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:40.296519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-feba7099-a5aa-4247-92bd-8e8bfc48b3b9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:40.297737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-feba7099-a5aa-4247-92bd-8e8bfc48b3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:40.359686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:40.405903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 666/1322] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:40.465432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04f9e-ded6-477c-b7ab-6bc610de89bd None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:40.465713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04f9e-ded6-477c-b7ab-6bc610de89bd None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:40.465987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb04f9e-ded6-477c-b7ab-6bc610de89bd None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:40.467517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 657/1323] 10.222.0.50 () {54 vars in 964 bytes} [Thu Jun 30 15:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:40.474474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680e6f-ff56-489c-b13c-26b8e969ab03 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:40.474673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680e6f-ff56-489c-b13c-26b8e969ab03 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:40.474924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a680e6f-ff56-489c-b13c-26b8e969ab03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:40.562002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7a680e6f-ff56-489c-b13c-26b8e969ab03 None None] MFA Rules not processed for user `5e17d25a83fe48d99ddbd209fbea8486`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:40.564149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7a680e6f-ff56-489c-b13c-26b8e969ab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:40.645302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 667/1324] 10.222.0.50 () {58 vars in 1061 bytes} [Thu Jun 30 15:42:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:40.654565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb04f9e-ded6-477c-b7ab-6bc610de89bd None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:40.656152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-efb04f9e-ded6-477c-b7ab-6bc610de89bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:40.676777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:40.766221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb04f9e-ded6-477c-b7ab-6bc610de89bd None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:40.841309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae748fe-739f-438d-ad1f-c3dee1d6f1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:40.841960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae748fe-739f-438d-ad1f-c3dee1d6f1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:40.842238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae748fe-739f-438d-ad1f-c3dee1d6f1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:40.842508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae748fe-739f-438d-ad1f-c3dee1d6f1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:40.844211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bae748fe-739f-438d-ad1f-c3dee1d6f1e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:40.920534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:40.958284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae748fe-739f-438d-ad1f-c3dee1d6f1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:40.959841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae748fe-739f-438d-ad1f-c3dee1d6f1e8 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:40.978701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 658/1325] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:42:43.260307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358777-9b13-4a80-9965-1a255fdf54b9 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:43.260875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358777-9b13-4a80-9965-1a255fdf54b9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:43.260875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358777-9b13-4a80-9965-1a255fdf54b9 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:43.262261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 668/1326] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:43.269339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07c4c-6077-4659-beff-ebc6525fd75d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:43.269339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07c4c-6077-4659-beff-ebc6525fd75d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:43.269339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d07c4c-6077-4659-beff-ebc6525fd75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:43.338132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-00d07c4c-6077-4659-beff-ebc6525fd75d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:43.339371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-00d07c4c-6077-4659-beff-ebc6525fd75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:43.450203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 659/1327] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:43.486005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf358777-9b13-4a80-9965-1a255fdf54b9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:43.487586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-cf358777-9b13-4a80-9965-1a255fdf54b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:43.548654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:43.592910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf358777-9b13-4a80-9965-1a255fdf54b9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:43.670129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e88615-f71f-477f-9366-5bb30ce7e5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:43.670696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e88615-f71f-477f-9366-5bb30ce7e5ec service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:43.671007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e88615-f71f-477f-9366-5bb30ce7e5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:43.671230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e88615-f71f-477f-9366-5bb30ce7e5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:43.673090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f4e88615-f71f-477f-9366-5bb30ce7e5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:43.787008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:43.789694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e88615-f71f-477f-9366-5bb30ce7e5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:43.791153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e88615-f71f-477f-9366-5bb30ce7e5ec service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:43.809499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 669/1328] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:42:45.932846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec1c9b-96ed-45dd-b5ce-ae965e26f923 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:45.933389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec1c9b-96ed-45dd-b5ce-ae965e26f923 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:45.934691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec1c9b-96ed-45dd-b5ce-ae965e26f923 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:45.936188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 660/1329] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:45.941866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24e30c-15ef-495f-9416-b7edc5d72374 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:45.942323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24e30c-15ef-495f-9416-b7edc5d72374 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:45.942417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24e30c-15ef-495f-9416-b7edc5d72374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:46.036935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-da24e30c-15ef-495f-9416-b7edc5d72374 None None] MFA Rules not processed for user `563a6491794745e8b41b36d0b205e9df`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:46.038159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-da24e30c-15ef-495f-9416-b7edc5d72374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:46.151602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 670/1330] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:46.193378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec1c9b-96ed-45dd-b5ce-ae965e26f923 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:46.195850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-07ec1c9b-96ed-45dd-b5ce-ae965e26f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:46.232158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:46.309821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec1c9b-96ed-45dd-b5ce-ae965e26f923 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:46.392370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b9576-f79b-421d-8078-360c3965c17e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:46.393041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b9576-f79b-421d-8078-360c3965c17e service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:46.393317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b9576-f79b-421d-8078-360c3965c17e service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:46.393581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b9576-f79b-421d-8078-360c3965c17e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:46.395240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a18b9576-f79b-421d-8078-360c3965c17e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:46.479282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:46.513301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b9576-f79b-421d-8078-360c3965c17e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:46.514878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b9576-f79b-421d-8078-360c3965c17e service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:46.533839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 661/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:42:48.973213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90434783-2294-4760-9a17-fed957fae8bc None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:48.974234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90434783-2294-4760-9a17-fed957fae8bc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:48.974234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90434783-2294-4760-9a17-fed957fae8bc None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:48.976485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 671/1332] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:48.985679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23f73-47ba-4b51-a909-3dd8ece9ce3c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:48.986119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23f73-47ba-4b51-a909-3dd8ece9ce3c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:48.986281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c23f73-47ba-4b51-a909-3dd8ece9ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:49.077894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f6c23f73-47ba-4b51-a909-3dd8ece9ce3c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:49.079131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f6c23f73-47ba-4b51-a909-3dd8ece9ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:49.180102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 662/1333] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:49.222279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90434783-2294-4760-9a17-fed957fae8bc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:49.223942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-90434783-2294-4760-9a17-fed957fae8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:49.306351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:49.340866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90434783-2294-4760-9a17-fed957fae8bc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:49.421700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c03c75a-8c85-4b92-a11d-8f46fb9266a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:49.422352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c75a-8c85-4b92-a11d-8f46fb9266a8 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:49.422632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c75a-8c85-4b92-a11d-8f46fb9266a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:49.422905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c03c75a-8c85-4b92-a11d-8f46fb9266a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:49.424602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6c03c75a-8c85-4b92-a11d-8f46fb9266a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:49.541919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03c75a-8c85-4b92-a11d-8f46fb9266a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:49.543409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c03c75a-8c85-4b92-a11d-8f46fb9266a8 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:49.553874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:49.564561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 672/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:42:51.667827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833da67-8278-48b6-a8fb-5944776db2a8 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:51.668432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833da67-8278-48b6-a8fb-5944776db2a8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:51.668432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833da67-8278-48b6-a8fb-5944776db2a8 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:51.670167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 663/1335] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:51.676666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c7055-b464-4fd2-8c57-a2ac5b93fdb9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:51.677090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c7055-b464-4fd2-8c57-a2ac5b93fdb9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:51.677237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c7055-b464-4fd2-8c57-a2ac5b93fdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:51.760741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a96c7055-b464-4fd2-8c57-a2ac5b93fdb9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:51.761960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a96c7055-b464-4fd2-8c57-a2ac5b93fdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:51.860342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 673/1336] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:51.904307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833da67-8278-48b6-a8fb-5944776db2a8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:51.905866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c833da67-8278-48b6-a8fb-5944776db2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:51.950834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:52.023185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833da67-8278-48b6-a8fb-5944776db2a8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:52.109598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d157a-4d8a-4f54-982f-6f3c4b28d425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:52.110244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d157a-4d8a-4f54-982f-6f3c4b28d425 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:52.110531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d157a-4d8a-4f54-982f-6f3c4b28d425 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:52.110796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d157a-4d8a-4f54-982f-6f3c4b28d425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:52.112484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d92d157a-4d8a-4f54-982f-6f3c4b28d425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:52.201793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:52.229812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d157a-4d8a-4f54-982f-6f3c4b28d425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:52.231259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d157a-4d8a-4f54-982f-6f3c4b28d425 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:52.251679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 664/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:42:54.741102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3714b-ebf5-407c-8473-29349599b43e None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:54.741697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3714b-ebf5-407c-8473-29349599b43e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:54.741697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3714b-ebf5-407c-8473-29349599b43e None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:54.743066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 674/1338] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:54.749905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6676a59-4fd1-40ed-baeb-a374c7d017bf None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:54.750653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6676a59-4fd1-40ed-baeb-a374c7d017bf None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:54.750984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6676a59-4fd1-40ed-baeb-a374c7d017bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:54.820894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a6676a59-4fd1-40ed-baeb-a374c7d017bf None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:54.822088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a6676a59-4fd1-40ed-baeb-a374c7d017bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:54.937548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 665/1339] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:54.988683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3714b-ebf5-407c-8473-29349599b43e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:54.991080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fcd3714b-ebf5-407c-8473-29349599b43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:55.077804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:55.109410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3714b-ebf5-407c-8473-29349599b43e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:55.190096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab888323-1cd7-49a0-ba4f-f72082abc1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:55.190751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab888323-1cd7-49a0-ba4f-f72082abc1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:55.191025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab888323-1cd7-49a0-ba4f-f72082abc1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:55.191297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab888323-1cd7-49a0-ba4f-f72082abc1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:55.193007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ab888323-1cd7-49a0-ba4f-f72082abc1a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:55.318758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab888323-1cd7-49a0-ba4f-f72082abc1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:55.320250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab888323-1cd7-49a0-ba4f-f72082abc1a6 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:55.341237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 675/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:42:59.367153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e613e-27fe-4959-8ffb-46d6a731f33f None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:59.367688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e613e-27fe-4959-8ffb-46d6a731f33f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:59.367688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e613e-27fe-4959-8ffb-46d6a731f33f None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:59.369372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 666/1341] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:42:59.375105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540031-19a5-4532-b98a-0e944fc36123 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:59.375555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540031-19a5-4532-b98a-0e944fc36123 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:59.375664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540031-19a5-4532-b98a-0e944fc36123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:59.382623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:59.451512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-dc540031-19a5-4532-b98a-0e944fc36123 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:42:59.452797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-dc540031-19a5-4532-b98a-0e944fc36123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:59.565237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 676/1342] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:42:59.604386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e613e-27fe-4959-8ffb-46d6a731f33f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:42:59.606821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-752e613e-27fe-4959-8ffb-46d6a731f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:42:59.659731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:42:59.708328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e613e-27fe-4959-8ffb-46d6a731f33f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:42:59.712071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873791-0c23-4739-9168-5a991f6d72c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:42:59.712850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873791-0c23-4739-9168-5a991f6d72c5 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:42:59.713131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873791-0c23-4739-9168-5a991f6d72c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:42:59.713401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873791-0c23-4739-9168-5a991f6d72c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:42:59.722797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873791-0c23-4739-9168-5a991f6d72c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:42:59.724258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873791-0c23-4739-9168-5a991f6d72c5 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:42:59.733477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 667/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:02.486116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62093c87-1630-4310-ac03-0f1d01ccd112 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:02.486696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62093c87-1630-4310-ac03-0f1d01ccd112 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:02.486696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62093c87-1630-4310-ac03-0f1d01ccd112 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:02.488065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 677/1344] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:02.494361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58789133-4165-4d31-a2b1-71d529e35988 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:02.494930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58789133-4165-4d31-a2b1-71d529e35988 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:02.494930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58789133-4165-4d31-a2b1-71d529e35988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:02.525885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-58789133-4165-4d31-a2b1-71d529e35988 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:02.527005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-58789133-4165-4d31-a2b1-71d529e35988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:02.610169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 668/1345] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:02.647871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62093c87-1630-4310-ac03-0f1d01ccd112 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:02.659249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62093c87-1630-4310-ac03-0f1d01ccd112 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:02.662976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509d1d6-c945-45f3-9d5c-3f805d6c26ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:02.663705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d1d6-c945-45f3-9d5c-3f805d6c26ad service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:02.664081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d1d6-c945-45f3-9d5c-3f805d6c26ad service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:02.664413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d1d6-c945-45f3-9d5c-3f805d6c26ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:02.673827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509d1d6-c945-45f3-9d5c-3f805d6c26ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:02.675270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509d1d6-c945-45f3-9d5c-3f805d6c26ad service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:02.684891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 678/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:06.889016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4d1ae-44f6-498c-ab29-1bbd51c6003a None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:06.889573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4d1ae-44f6-498c-ab29-1bbd51c6003a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:06.889573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c4d1ae-44f6-498c-ab29-1bbd51c6003a None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:06.891075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 669/1347] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:06.896656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3d92-ff5f-4dff-a583-355cb42967bb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:06.897141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3d92-ff5f-4dff-a583-355cb42967bb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:06.897235 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3d92-ff5f-4dff-a583-355cb42967bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:06.916550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-421a3d92-ff5f-4dff-a583-355cb42967bb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:06.917857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-421a3d92-ff5f-4dff-a583-355cb42967bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:06.928043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 679/1348] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:43:06.964834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4d1ae-44f6-498c-ab29-1bbd51c6003a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:06.967023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-43c4d1ae-44f6-498c-ab29-1bbd51c6003a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:06.980541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c4d1ae-44f6-498c-ab29-1bbd51c6003a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:06.984242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bd528-e7d2-4347-8b0f-fdff286747d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:06.984955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bd528-e7d2-4347-8b0f-fdff286747d4 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:06.985303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bd528-e7d2-4347-8b0f-fdff286747d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:06.985556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bd528-e7d2-4347-8b0f-fdff286747d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:06.995251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bd528-e7d2-4347-8b0f-fdff286747d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:06.996800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50bd528-e7d2-4347-8b0f-fdff286747d4 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:07.006730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 670/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:12.053777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1056-c1cc-4de9-9b3d-42ae73cd4425 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:12.054294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1056-c1cc-4de9-9b3d-42ae73cd4425 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:12.054509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb1056-c1cc-4de9-9b3d-42ae73cd4425 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:12.072220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 680/1350] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:43:12 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:12.072437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23e8b8-727d-413e-8f60-9e3f3a79d8eb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:12.072869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23e8b8-727d-413e-8f60-9e3f3a79d8eb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:12.072958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23e8b8-727d-413e-8f60-9e3f3a79d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:12.211587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-de23e8b8-727d-413e-8f60-9e3f3a79d8eb None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:12.212855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-de23e8b8-727d-413e-8f60-9e3f3a79d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:12.244777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:43:12.292167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 671/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:43:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:43:16.376447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b937fc-85c0-4638-b42a-df90b78ca07b None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:16.377037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b937fc-85c0-4638-b42a-df90b78ca07b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:16.377291 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b937fc-85c0-4638-b42a-df90b78ca07b None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:16.378924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 681/1352] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:43:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:16.384519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d45e-15e7-4884-bfcc-4131a4a5932c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:16.384987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d45e-15e7-4884-bfcc-4131a4a5932c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:16.385078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9d45e-15e7-4884-bfcc-4131a4a5932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:16.403732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fea9d45e-15e7-4884-bfcc-4131a4a5932c None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:16.404937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fea9d45e-15e7-4884-bfcc-4131a4a5932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:16.413777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 672/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:43:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:43:16.911547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=72 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:43:17.950132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc05bff-fee4-4e30-9580-1e50ae4fa4b5 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:17.950742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc05bff-fee4-4e30-9580-1e50ae4fa4b5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:17.950742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc05bff-fee4-4e30-9580-1e50ae4fa4b5 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:17.952128 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 682/1354] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:17.956971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4acc-8aa3-41fb-9cb9-07b7700ba171 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:17.957417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4acc-8aa3-41fb-9cb9-07b7700ba171 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:17.957662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4acc-8aa3-41fb-9cb9-07b7700ba171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:17.976347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-984b4acc-8aa3-41fb-9cb9-07b7700ba171 None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:17.977505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-984b4acc-8aa3-41fb-9cb9-07b7700ba171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:17.986852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 673/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:43:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:21.500140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea432-9474-4321-a76f-ab0c241aab33 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:21.500140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea432-9474-4321-a76f-ab0c241aab33 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:21.500140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea432-9474-4321-a76f-ab0c241aab33 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:21.502717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 683/1356] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:21.507437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ed439-ca05-40c8-8058-49632586f70e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:21.507926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ed439-ca05-40c8-8058-49632586f70e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:21.508230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ed439-ca05-40c8-8058-49632586f70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:21.529164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7a4ed439-ca05-40c8-8058-49632586f70e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:21.531668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7a4ed439-ca05-40c8-8058-49632586f70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:21.546066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 674/1357] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:21.601252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46dd2e-6bc4-4259-a084-f84ab6a01394 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:21.601765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46dd2e-6bc4-4259-a084-f84ab6a01394 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:21.602202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46dd2e-6bc4-4259-a084-f84ab6a01394 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:21.604135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 684/1358] 10.222.0.50 () {54 vars in 961 bytes} [Thu Jun 30 15:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:21.612653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbb5ba-1f35-40d6-a777-0d49722a258d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:21.614165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbb5ba-1f35-40d6-a777-0d49722a258d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:21.614165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cbb5ba-1f35-40d6-a777-0d49722a258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:21.675532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-73cbb5ba-1f35-40d6-a777-0d49722a258d None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:21.676867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-73cbb5ba-1f35-40d6-a777-0d49722a258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:21.771295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 675/1359] 10.222.0.50 () {58 vars in 1058 bytes} [Thu Jun 30 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:43:21.779321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46dd2e-6bc4-4259-a084-f84ab6a01394 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:21.794852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46dd2e-6bc4-4259-a084-f84ab6a01394 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:21.801140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36906006-6dde-4de3-873f-db0f03cfb4de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:21.801993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906006-6dde-4de3-873f-db0f03cfb4de service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:21.802447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906006-6dde-4de3-873f-db0f03cfb4de service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:21.803014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36906006-6dde-4de3-873f-db0f03cfb4de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:21.812771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36906006-6dde-4de3-873f-db0f03cfb4de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:21.814627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36906006-6dde-4de3-873f-db0f03cfb4de service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:21.825752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 685/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:24.049814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38c58e-0c27-409f-bb57-5505ce813a7c None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:24.050392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38c58e-0c27-409f-bb57-5505ce813a7c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:24.050864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38c58e-0c27-409f-bb57-5505ce813a7c None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:24.052645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 676/1361] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:24.058381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d7be5-f7a0-4aa9-8351-6cf612b3c576 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:24.059239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d7be5-f7a0-4aa9-8351-6cf612b3c576 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:24.059697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d7be5-f7a0-4aa9-8351-6cf612b3c576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:24.078109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-971d7be5-f7a0-4aa9-8351-6cf612b3c576 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:24.079299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-971d7be5-f7a0-4aa9-8351-6cf612b3c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:24.087158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 686/1362] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:24.131108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38c58e-0c27-409f-bb57-5505ce813a7c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:24.146227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38c58e-0c27-409f-bb57-5505ce813a7c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:24.151701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d119108-2d88-43e4-a7cd-17319f455f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:24.152394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119108-2d88-43e4-a7cd-17319f455f81 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:24.152694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119108-2d88-43e4-a7cd-17319f455f81 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:24.152967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119108-2d88-43e4-a7cd-17319f455f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:24.161780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d119108-2d88-43e4-a7cd-17319f455f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:24.163373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d119108-2d88-43e4-a7cd-17319f455f81 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:24.173531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 677/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:26.511585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740103e7-ed61-4462-8703-d1b5a33272a5 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:26.512610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740103e7-ed61-4462-8703-d1b5a33272a5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:26.513102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740103e7-ed61-4462-8703-d1b5a33272a5 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:26.515045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 687/1364] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:26.520845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1fa6-748e-4309-9f60-ae96829d78cc None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:26.521301 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1fa6-748e-4309-9f60-ae96829d78cc None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:26.521398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf1fa6-748e-4309-9f60-ae96829d78cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:26.539438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d5bf1fa6-748e-4309-9f60-ae96829d78cc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:26.540888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d5bf1fa6-748e-4309-9f60-ae96829d78cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:26.548900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 678/1365] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:26.594580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069f488-391d-4877-8d1a-ef51214d08e4 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:26.594880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069f488-391d-4877-8d1a-ef51214d08e4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:26.595216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069f488-391d-4877-8d1a-ef51214d08e4 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:26.596629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 688/1366] 10.222.0.50 () {54 vars in 961 bytes} [Thu Jun 30 15:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:26.602159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46b22-ba91-413a-9c46-6b28804f33c6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:26.602447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46b22-ba91-413a-9c46-6b28804f33c6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:26.602723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed46b22-ba91-413a-9c46-6b28804f33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:26.621277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3ed46b22-ba91-413a-9c46-6b28804f33c6 None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:26.622468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3ed46b22-ba91-413a-9c46-6b28804f33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:26.631846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 679/1367] 10.222.0.50 () {58 vars in 1058 bytes} [Thu Jun 30 15:43:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:26.639754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069f488-391d-4877-8d1a-ef51214d08e4 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:26.654414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069f488-391d-4877-8d1a-ef51214d08e4 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:26.660189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9bd7a-7d81-4a43-a1c4-52587a96badd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:26.660815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9bd7a-7d81-4a43-a1c4-52587a96badd service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:26.661096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9bd7a-7d81-4a43-a1c4-52587a96badd service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:26.661369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9bd7a-7d81-4a43-a1c4-52587a96badd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:26.669732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9bd7a-7d81-4a43-a1c4-52587a96badd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:26.671304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9bd7a-7d81-4a43-a1c4-52587a96badd service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:26.680577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 689/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:28.779701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190f2b6-b267-467a-b89e-74e56e7db596 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:28.780310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190f2b6-b267-467a-b89e-74e56e7db596 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:28.780310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1190f2b6-b267-467a-b89e-74e56e7db596 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:28.781666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 680/1369] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:43:28.787687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5342882-61f9-45eb-98bf-f351e16f961b None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:28.788347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5342882-61f9-45eb-98bf-f351e16f961b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:28.788463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5342882-61f9-45eb-98bf-f351e16f961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:28.807875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a5342882-61f9-45eb-98bf-f351e16f961b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:28.809466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a5342882-61f9-45eb-98bf-f351e16f961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:28.818575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 690/1370] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:28.862453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190f2b6-b267-467a-b89e-74e56e7db596 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:28.877255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1190f2b6-b267-467a-b89e-74e56e7db596 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:28.882859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faa111-55f4-45c5-9ac7-836f5caecacc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:28.883766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa111-55f4-45c5-9ac7-836f5caecacc service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:28.884315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa111-55f4-45c5-9ac7-836f5caecacc service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:28.884795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faa111-55f4-45c5-9ac7-836f5caecacc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:28.893990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faa111-55f4-45c5-9ac7-836f5caecacc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:28.895834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faa111-55f4-45c5-9ac7-836f5caecacc service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:28.906032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 681/1371] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:29.450680 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276026ee-5891-4d5a-b0c8-4a9766a06748 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:29.450936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276026ee-5891-4d5a-b0c8-4a9766a06748 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:29.451189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276026ee-5891-4d5a-b0c8-4a9766a06748 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:29.452808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 691/1372] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:29.458517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ebadc-e7f1-4f4e-9e81-b48bd95e3144 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:29.458809 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ebadc-e7f1-4f4e-9e81-b48bd95e3144 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:29.459086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ebadc-e7f1-4f4e-9e81-b48bd95e3144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:29.484278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-2f1ebadc-e7f1-4f4e-9e81-b48bd95e3144 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:29.490270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2f1ebadc-e7f1-4f4e-9e81-b48bd95e3144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:29.501474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 682/1373] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:29.841438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138c700-b0a2-4741-948b-0999bcbd906a None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:29.841438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138c700-b0a2-4741-948b-0999bcbd906a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:29.841438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9138c700-b0a2-4741-948b-0999bcbd906a None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:29.843167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 692/1374] 10.222.0.50 () {54 vars in 963 bytes} [Thu Jun 30 15:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:29.849058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fab5e-766d-4764-825f-992c4392628b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:29.849058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fab5e-766d-4764-825f-992c4392628b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:29.849058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fab5e-766d-4764-825f-992c4392628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:29.856553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:43:29.900114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d95fab5e-766d-4764-825f-992c4392628b None None] MFA Rules not processed for user `7c9cdda3d7884298a791f90523f3ab91`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:29.901305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d95fab5e-766d-4764-825f-992c4392628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:29.962483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 683/1375] 10.222.0.50 () {58 vars in 1060 bytes} [Thu Jun 30 15:43:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:43:29.970050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138c700-b0a2-4741-948b-0999bcbd906a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:29.981875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9138c700-b0a2-4741-948b-0999bcbd906a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:29.985487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09199b32-6eeb-4261-b43e-db63e4dce4a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:29.986212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199b32-6eeb-4261-b43e-db63e4dce4a2 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:29.986501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199b32-6eeb-4261-b43e-db63e4dce4a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:29.986763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09199b32-6eeb-4261-b43e-db63e4dce4a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:29.996969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199b32-6eeb-4261-b43e-db63e4dce4a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:29.998591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09199b32-6eeb-4261-b43e-db63e4dce4a2 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:30.008587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 693/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:30.951816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb365c-815c-4bd7-8bb8-5ed2cf9b1a03 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:30.952472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb365c-815c-4bd7-8bb8-5ed2cf9b1a03 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:30.952472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb365c-815c-4bd7-8bb8-5ed2cf9b1a03 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:30.953796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 684/1377] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:30.959115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa6b29-7f2e-4906-832a-ca49c8ae68a1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:30.959541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa6b29-7f2e-4906-832a-ca49c8ae68a1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:30.959664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa6b29-7f2e-4906-832a-ca49c8ae68a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:30.978958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-08aa6b29-7f2e-4906-832a-ca49c8ae68a1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:30.980243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-08aa6b29-7f2e-4906-832a-ca49c8ae68a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:30.988576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 694/1378] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:31.033224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb365c-815c-4bd7-8bb8-5ed2cf9b1a03 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:31.050600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb365c-815c-4bd7-8bb8-5ed2cf9b1a03 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:31.056868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c03f8a-6035-43ed-afc3-66595e4a6650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:31.057526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c03f8a-6035-43ed-afc3-66595e4a6650 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:31.057808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c03f8a-6035-43ed-afc3-66595e4a6650 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:31.058084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c03f8a-6035-43ed-afc3-66595e4a6650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:31.066670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c03f8a-6035-43ed-afc3-66595e4a6650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:31.068322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c03f8a-6035-43ed-afc3-66595e4a6650 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:31.077988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 685/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:33.140485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041bfc4-7eb9-4fe0-b445-ed32516595e4 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:33.141062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041bfc4-7eb9-4fe0-b445-ed32516595e4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:33.141062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041bfc4-7eb9-4fe0-b445-ed32516595e4 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:33.142371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 695/1380] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:33.147968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f684f7-34b5-4f6d-b539-a51e98790600 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:33.148407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f684f7-34b5-4f6d-b539-a51e98790600 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:33.148593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f684f7-34b5-4f6d-b539-a51e98790600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:33.167712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-92f684f7-34b5-4f6d-b539-a51e98790600 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:33.168961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-92f684f7-34b5-4f6d-b539-a51e98790600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:33.177942 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 686/1381] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:33.222265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041bfc4-7eb9-4fe0-b445-ed32516595e4 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:33.237659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041bfc4-7eb9-4fe0-b445-ed32516595e4 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:33.244232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d3ecc-2b0a-45c6-b3e6-2e38afe453fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:33.244924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d3ecc-2b0a-45c6-b3e6-2e38afe453fb service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:33.245214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d3ecc-2b0a-45c6-b3e6-2e38afe453fb service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:33.245488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d3ecc-2b0a-45c6-b3e6-2e38afe453fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:33.254183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d3ecc-2b0a-45c6-b3e6-2e38afe453fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:33.255795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d3ecc-2b0a-45c6-b3e6-2e38afe453fb service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:33.272064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 696/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:35.339934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3ecca-005c-417b-90c9-fecfc338c225 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:35.340611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3ecca-005c-417b-90c9-fecfc338c225 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:35.340611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3ecca-005c-417b-90c9-fecfc338c225 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:35.341931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 687/1383] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:35.347580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c8c3c-7ec8-4772-8174-f415e0313c38 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:35.348068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c8c3c-7ec8-4772-8174-f415e0313c38 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:35.348174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c8c3c-7ec8-4772-8174-f415e0313c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:35.366623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ee0c8c3c-7ec8-4772-8174-f415e0313c38 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:35.367816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ee0c8c3c-7ec8-4772-8174-f415e0313c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:35.376457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 697/1384] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:35.419661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3ecca-005c-417b-90c9-fecfc338c225 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:35.436320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3ecca-005c-417b-90c9-fecfc338c225 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:35.443305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03c19e-2504-46cd-b762-bbf38f617968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:35.444151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c19e-2504-46cd-b762-bbf38f617968 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:35.444568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c19e-2504-46cd-b762-bbf38f617968 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:35.444950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03c19e-2504-46cd-b762-bbf38f617968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:35.456270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03c19e-2504-46cd-b762-bbf38f617968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:35.458262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03c19e-2504-46cd-b762-bbf38f617968 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:35.472929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 688/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:37.515541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84706f-1b2b-46f7-940c-645b6c4d8050 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:37.516077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84706f-1b2b-46f7-940c-645b6c4d8050 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:37.516176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84706f-1b2b-46f7-940c-645b6c4d8050 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:37.517580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 698/1386] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:37.523538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79461e50-4b62-48af-a831-b5129d5d78e3 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:37.524044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79461e50-4b62-48af-a831-b5129d5d78e3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:37.524277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79461e50-4b62-48af-a831-b5129d5d78e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:37.546913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-79461e50-4b62-48af-a831-b5129d5d78e3 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:37.548480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-79461e50-4b62-48af-a831-b5129d5d78e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:37.558748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 689/1387] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:37.606784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84706f-1b2b-46f7-940c-645b6c4d8050 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:37.626886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a84706f-1b2b-46f7-940c-645b6c4d8050 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:37.633256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95b527-9863-4e5c-a439-e2db73b447dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:37.633945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95b527-9863-4e5c-a439-e2db73b447dd service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:37.634225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95b527-9863-4e5c-a439-e2db73b447dd service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:37.634504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95b527-9863-4e5c-a439-e2db73b447dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:37.643827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95b527-9863-4e5c-a439-e2db73b447dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:37.645554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95b527-9863-4e5c-a439-e2db73b447dd service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:37.655693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 699/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:39.977243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dea60d-f5f7-4d94-af6e-825da54ab616 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:39.977835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dea60d-f5f7-4d94-af6e-825da54ab616 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:39.977956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dea60d-f5f7-4d94-af6e-825da54ab616 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:39.979683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 690/1389] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:39.985271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1e02-f93c-4312-952f-cba9bae77b4f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:39.985887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1e02-f93c-4312-952f-cba9bae77b4f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:39.985887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f1e02-f93c-4312-952f-cba9bae77b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:40.004556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-579f1e02-f93c-4312-952f-cba9bae77b4f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:40.005811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-579f1e02-f93c-4312-952f-cba9bae77b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:40.014755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 700/1390] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:40.062661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dea60d-f5f7-4d94-af6e-825da54ab616 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:40.080798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dea60d-f5f7-4d94-af6e-825da54ab616 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:40.088624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cc258-5792-4d01-8500-a57b5181c1a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:40.089412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cc258-5792-4d01-8500-a57b5181c1a8 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:40.089769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cc258-5792-4d01-8500-a57b5181c1a8 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:40.090303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cc258-5792-4d01-8500-a57b5181c1a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:40.101307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cc258-5792-4d01-8500-a57b5181c1a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:40.103100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cc258-5792-4d01-8500-a57b5181c1a8 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:40.114117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 691/1391] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:42.269712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ff81c-c259-4c71-ad73-a344bc71ef54 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:42.269712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ff81c-c259-4c71-ad73-a344bc71ef54 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:42.269712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ff81c-c259-4c71-ad73-a344bc71ef54 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:42.271006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 701/1392] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:42.276921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df628e-2e52-4b3b-8a46-0dff6e77d7ef None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:42.277362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df628e-2e52-4b3b-8a46-0dff6e77d7ef None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:42.277553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df628e-2e52-4b3b-8a46-0dff6e77d7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:42.297662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-98df628e-2e52-4b3b-8a46-0dff6e77d7ef None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:42.298984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-98df628e-2e52-4b3b-8a46-0dff6e77d7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:42.307543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 692/1393] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:42.353674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ff81c-c259-4c71-ad73-a344bc71ef54 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:42.371115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ff81c-c259-4c71-ad73-a344bc71ef54 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:42.377993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a8ea9-7dad-4225-b053-0a7e33e87dac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:42.378777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8ea9-7dad-4225-b053-0a7e33e87dac service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:42.379139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8ea9-7dad-4225-b053-0a7e33e87dac service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:42.379472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8ea9-7dad-4225-b053-0a7e33e87dac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:42.390794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8ea9-7dad-4225-b053-0a7e33e87dac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:42.393029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a8ea9-7dad-4225-b053-0a7e33e87dac service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:42.406054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 702/1394] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:44.558352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549b66-c81a-4a11-bb59-b545a6e5090a None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:44.558879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549b66-c81a-4a11-bb59-b545a6e5090a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:44.558879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2549b66-c81a-4a11-bb59-b545a6e5090a None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:44.560455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 693/1395] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:44.565842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb1c17-f292-4949-989f-4cf02e74c9e6 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:44.566382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb1c17-f292-4949-989f-4cf02e74c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:44.566382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb1c17-f292-4949-989f-4cf02e74c9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:44.584561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-bfdb1c17-f292-4949-989f-4cf02e74c9e6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:44.585761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bfdb1c17-f292-4949-989f-4cf02e74c9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:44.594344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 703/1396] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:44.639331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549b66-c81a-4a11-bb59-b545a6e5090a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:44.655614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2549b66-c81a-4a11-bb59-b545a6e5090a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:44.661493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1859cd-cd88-4ee2-8215-4f7727cf91e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:44.662133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1859cd-cd88-4ee2-8215-4f7727cf91e9 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:44.662418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1859cd-cd88-4ee2-8215-4f7727cf91e9 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:44.662694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1859cd-cd88-4ee2-8215-4f7727cf91e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:44.671266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1859cd-cd88-4ee2-8215-4f7727cf91e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:44.672953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1859cd-cd88-4ee2-8215-4f7727cf91e9 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:44.683329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 694/1397] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:45.007097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e828841-5974-49df-bb11-e490e4921ebe None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:45.007343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e828841-5974-49df-bb11-e490e4921ebe None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:45.007646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e828841-5974-49df-bb11-e490e4921ebe None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:45.009068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 704/1398] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:45.014347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cc984-a6fe-418f-8c62-a99f95eff4dd None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:45.014711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cc984-a6fe-418f-8c62-a99f95eff4dd None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:45.014969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cc984-a6fe-418f-8c62-a99f95eff4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:45.034269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-2e8cc984-a6fe-418f-8c62-a99f95eff4dd None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:45.035662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2e8cc984-a6fe-418f-8c62-a99f95eff4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:45.043948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 695/1399] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:45.100348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4132e13-79bd-4307-9a2d-505efe3ea052 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:45.100610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4132e13-79bd-4307-9a2d-505efe3ea052 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:45.100943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4132e13-79bd-4307-9a2d-505efe3ea052 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:45.102288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 705/1400] 10.222.0.50 () {54 vars in 962 bytes} [Thu Jun 30 15:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:45.107450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eb36f-4518-421c-8240-9eeee9900fe2 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:45.107835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eb36f-4518-421c-8240-9eeee9900fe2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:45.108191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eb36f-4518-421c-8240-9eeee9900fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:45.157380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9f5eb36f-4518-421c-8240-9eeee9900fe2 None None] MFA Rules not processed for user `e4cb4d5213c647459e1c21c311a4a305`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:45.158566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9f5eb36f-4518-421c-8240-9eeee9900fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:45.191092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:43:45.224201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 696/1401] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Jun 30 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:45.232362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4132e13-79bd-4307-9a2d-505efe3ea052 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:45.243653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4132e13-79bd-4307-9a2d-505efe3ea052 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:45.247827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb9915-bce2-4ee7-b306-f6b8df0c4f31 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:45.248511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9915-bce2-4ee7-b306-f6b8df0c4f31 service glance] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:45.248788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9915-bce2-4ee7-b306-f6b8df0c4f31 service glance] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:45.249056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb9915-bce2-4ee7-b306-f6b8df0c4f31 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:45.258493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb9915-bce2-4ee7-b306-f6b8df0c4f31 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:45.259850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb9915-bce2-4ee7-b306-f6b8df0c4f31 service glance] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:45.269975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 706/1402] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:43:45.987713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bba19-41c2-4e62-9292-ebb2195955c1 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:45.988241 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bba19-41c2-4e62-9292-ebb2195955c1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:45.988330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bba19-41c2-4e62-9292-ebb2195955c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:46.035781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e77bba19-41c2-4e62-9292-ebb2195955c1 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:46.037029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e77bba19-41c2-4e62-9292-ebb2195955c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:46.107311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 697/1403] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:46.152886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c83dd-737c-484a-9426-796a9a549045 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:46.153354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c83dd-737c-484a-9426-796a9a549045 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:46.153584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c83dd-737c-484a-9426-796a9a549045 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:46.155045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 707/1404] 10.222.0.50 () {54 vars in 942 bytes} [Thu Jun 30 15:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:43:46.161705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccaaf5-f262-4c6a-b8f7-173fced76bfb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:46.162014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccaaf5-f262-4c6a-b8f7-173fced76bfb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:46.162293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccaaf5-f262-4c6a-b8f7-173fced76bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:46.211850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-84ccaaf5-f262-4c6a-b8f7-173fced76bfb None None] MFA Rules not processed for user `2933f88027fc435eba3064124e2dd5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:46.212958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-84ccaaf5-f262-4c6a-b8f7-173fced76bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:46.294343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 698/1405] 10.222.0.50 () {58 vars in 1039 bytes} [Thu Jun 30 15:43:46 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:46.305599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c83dd-737c-484a-9426-796a9a549045 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:46.323720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c83dd-737c-484a-9426-796a9a549045 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:46.329222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bc13d-9207-47f9-9020-2e13f34460d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:46.329869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bc13d-9207-47f9-9020-2e13f34460d7 service swift] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:46.330148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bc13d-9207-47f9-9020-2e13f34460d7 service swift] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:46.330428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bc13d-9207-47f9-9020-2e13f34460d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:46.341405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bc13d-9207-47f9-9020-2e13f34460d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:46.343281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bc13d-9207-47f9-9020-2e13f34460d7 service swift] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:46.353776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 708/1406] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:43:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:43:46.796081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a4fd1-c0ef-4023-9ef4-a6ffe78e51cb None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:46.796452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a4fd1-c0ef-4023-9ef4-a6ffe78e51cb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:46.796566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a4fd1-c0ef-4023-9ef4-a6ffe78e51cb None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:46.797991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 699/1407] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:43:46.803642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534d98c-87e7-448e-ace1-2a06e49c62b1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:46.804178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534d98c-87e7-448e-ace1-2a06e49c62b1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:46.804455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534d98c-87e7-448e-ace1-2a06e49c62b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:46.824917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5534d98c-87e7-448e-ace1-2a06e49c62b1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:43:46.826378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5534d98c-87e7-448e-ace1-2a06e49c62b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:43:46.834751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 709/1408] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:43:46.881306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a4fd1-c0ef-4023-9ef4-a6ffe78e51cb None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:43:46.903405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a4fd1-c0ef-4023-9ef4-a6ffe78e51cb None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:43:46.912500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2b037-679e-47db-a7e9-51a3703f1201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:43:46.913583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b037-679e-47db-a7e9-51a3703f1201 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:43:46.914041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b037-679e-47db-a7e9-51a3703f1201 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:43:46.914326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b037-679e-47db-a7e9-51a3703f1201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:43:46.924555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2b037-679e-47db-a7e9-51a3703f1201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:43:46.926187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2b037-679e-47db-a7e9-51a3703f1201 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:43:46.931960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:43:46.937577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 700/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:43:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:43:56.817201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=69 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:44:08.391653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f679e0-1256-47fe-a7ae-db076bcced99 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:08.392365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f679e0-1256-47fe-a7ae-db076bcced99 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:08.392365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f679e0-1256-47fe-a7ae-db076bcced99 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:08.393690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 710/1410] 10.222.0.50 () {54 vars in 918 bytes} [Thu Jun 30 15:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:08.399143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173e7c0-a774-4bce-9be5-ec7e26a922a7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:08.399558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173e7c0-a774-4bce-9be5-ec7e26a922a7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:08.399694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d173e7c0-a774-4bce-9be5-ec7e26a922a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:08.419959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d173e7c0-a774-4bce-9be5-ec7e26a922a7 None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:08.421182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d173e7c0-a774-4bce-9be5-ec7e26a922a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:08.431425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 701/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Jun 30 15:44:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:08.446113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8822e-1239-4ac1-a60d-7664d8e3651d None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:08.446406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8822e-1239-4ac1-a60d-7664d8e3651d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:08.446686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8822e-1239-4ac1-a60d-7664d8e3651d None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:08.448109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 711/1412] 10.222.0.50 () {54 vars in 918 bytes} [Thu Jun 30 15:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:08.452965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fae3be-1f7b-47b7-91ba-eca16e8e944d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:08.453252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fae3be-1f7b-47b7-91ba-eca16e8e944d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:08.453552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fae3be-1f7b-47b7-91ba-eca16e8e944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:08.471664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-97fae3be-1f7b-47b7-91ba-eca16e8e944d None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:08.472866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-97fae3be-1f7b-47b7-91ba-eca16e8e944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:08.482732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 702/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Thu Jun 30 15:44:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:10.976365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19d2b-4ca8-4843-9e79-910346fa4b59 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:10.977227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19d2b-4ca8-4843-9e79-910346fa4b59 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:10.977227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19d2b-4ca8-4843-9e79-910346fa4b59 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:10.979414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:10.986576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe67030-d341-4c76-bce6-da9b91bea664 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:10.987029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe67030-d341-4c76-bce6-da9b91bea664 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:10.987132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe67030-d341-4c76-bce6-da9b91bea664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:11.008132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-2fe67030-d341-4c76-bce6-da9b91bea664 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:11.009318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2fe67030-d341-4c76-bce6-da9b91bea664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:11.017951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:11.078640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe19d2b-4ca8-4843-9e79-910346fa4b59 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:11.095277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe19d2b-4ca8-4843-9e79-910346fa4b59 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:11.101152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b865b3e-ee6f-45b3-88ac-af9628730bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:11.101792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865b3e-ee6f-45b3-88ac-af9628730bb7 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:11.102068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865b3e-ee6f-45b3-88ac-af9628730bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:11.102337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b865b3e-ee6f-45b3-88ac-af9628730bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:11.110733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b865b3e-ee6f-45b3-88ac-af9628730bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:11.112339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b865b3e-ee6f-45b3-88ac-af9628730bb7 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:11.121989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 713/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:12.025381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee755078-b199-4789-a2df-0b1aff9ae986 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:12.025984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee755078-b199-4789-a2df-0b1aff9ae986 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:12.025984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee755078-b199-4789-a2df-0b1aff9ae986 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:12.027304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:12.032672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2f57d-b241-4717-984a-0c29f7e4eb34 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:12.033178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2f57d-b241-4717-984a-0c29f7e4eb34 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:12.033178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2f57d-b241-4717-984a-0c29f7e4eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:12.053653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-58d2f57d-b241-4717-984a-0c29f7e4eb34 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:12.062564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-58d2f57d-b241-4717-984a-0c29f7e4eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:12.076565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:12.228898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee755078-b199-4789-a2df-0b1aff9ae986 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:12.243870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee755078-b199-4789-a2df-0b1aff9ae986 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:12.249747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba72a41-dc1e-4b97-8997-91a91fcf7199 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:12.250402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba72a41-dc1e-4b97-8997-91a91fcf7199 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:12.250669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba72a41-dc1e-4b97-8997-91a91fcf7199 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:12.250943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba72a41-dc1e-4b97-8997-91a91fcf7199 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:12.259602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba72a41-dc1e-4b97-8997-91a91fcf7199 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:12.261336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba72a41-dc1e-4b97-8997-91a91fcf7199 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:12.272236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 705/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:13.155794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ed5a2-1f64-4a9e-8970-2b606fcfd350 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:13.156443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ed5a2-1f64-4a9e-8970-2b606fcfd350 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:13.156443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ed5a2-1f64-4a9e-8970-2b606fcfd350 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:13.157760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:13.163803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b3ff-c33a-4cfe-b909-8ad7fcb874ce None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:13.164306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b3ff-c33a-4cfe-b909-8ad7fcb874ce None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:13.164414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567b3ff-c33a-4cfe-b909-8ad7fcb874ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:13.186419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9567b3ff-c33a-4cfe-b909-8ad7fcb874ce None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:13.187625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9567b3ff-c33a-4cfe-b909-8ad7fcb874ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:13.196098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:13.350397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ed5a2-1f64-4a9e-8970-2b606fcfd350 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:13.370282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ed5a2-1f64-4a9e-8970-2b606fcfd350 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:13.376511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74948cb-6c1c-4eaa-9796-8f4a0556aa1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:13.377149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74948cb-6c1c-4eaa-9796-8f4a0556aa1b service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:13.377425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74948cb-6c1c-4eaa-9796-8f4a0556aa1b service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:13.377693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74948cb-6c1c-4eaa-9796-8f4a0556aa1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:13.386080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74948cb-6c1c-4eaa-9796-8f4a0556aa1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:13.387827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74948cb-6c1c-4eaa-9796-8f4a0556aa1b service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:13.398306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 716/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:14.268968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b50d-d1bd-4208-8d50-b009ab38f06f None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:14.269593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b50d-d1bd-4208-8d50-b009ab38f06f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:14.269593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b50d-d1bd-4208-8d50-b009ab38f06f None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:14.270960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:14.277641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0a9aa-15af-4e14-aabc-a7ecf3e1fec4 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:14.278388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0a9aa-15af-4e14-aabc-a7ecf3e1fec4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:14.278594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0a9aa-15af-4e14-aabc-a7ecf3e1fec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:14.299653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f6e0a9aa-15af-4e14-aabc-a7ecf3e1fec4 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:14.301788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f6e0a9aa-15af-4e14-aabc-a7ecf3e1fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:14.310970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:14.465638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b50d-d1bd-4208-8d50-b009ab38f06f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:14.483420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b50d-d1bd-4208-8d50-b009ab38f06f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:14.490402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29110de4-8503-45ea-b559-0899db8ba3cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:14.491030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29110de4-8503-45ea-b559-0899db8ba3cb service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:14.491311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29110de4-8503-45ea-b559-0899db8ba3cb service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:14.491581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29110de4-8503-45ea-b559-0899db8ba3cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:14.500604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29110de4-8503-45ea-b559-0899db8ba3cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:14.502197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29110de4-8503-45ea-b559-0899db8ba3cb service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:14.511937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 708/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:15.370275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce9581-1cfd-4e19-ad0f-c2d7ff97eae0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:15.371163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce9581-1cfd-4e19-ad0f-c2d7ff97eae0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:15.371163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce9581-1cfd-4e19-ad0f-c2d7ff97eae0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:15.373513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:15.383186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6484e1-6ba0-4f6e-bc3b-a47e552176a4 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:15.383630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6484e1-6ba0-4f6e-bc3b-a47e552176a4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:15.383884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6484e1-6ba0-4f6e-bc3b-a47e552176a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:15.404449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3c6484e1-6ba0-4f6e-bc3b-a47e552176a4 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:15.405676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3c6484e1-6ba0-4f6e-bc3b-a47e552176a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:15.414310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:15.565387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce9581-1cfd-4e19-ad0f-c2d7ff97eae0 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:15.586494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce9581-1cfd-4e19-ad0f-c2d7ff97eae0 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:15.592386 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d025b-f085-4b28-b24d-1b08eb98733c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:15.593200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d025b-f085-4b28-b24d-1b08eb98733c service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:15.593481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d025b-f085-4b28-b24d-1b08eb98733c service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:15.593754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d025b-f085-4b28-b24d-1b08eb98733c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:15.601945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d025b-f085-4b28-b24d-1b08eb98733c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:15.603545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d025b-f085-4b28-b24d-1b08eb98733c service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:15.613358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 719/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:16.436026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75e49c-baf2-4781-ba91-d17f5c050df9 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:16.438893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75e49c-baf2-4781-ba91-d17f5c050df9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:16.439205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75e49c-baf2-4781-ba91-d17f5c050df9 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:16.440674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:16 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:16.449505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a5c7c-b11a-4f05-b5ae-389a31d77273 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:16.450163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a5c7c-b11a-4f05-b5ae-389a31d77273 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:16.450396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a5c7c-b11a-4f05-b5ae-389a31d77273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:16.470644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f38a5c7c-b11a-4f05-b5ae-389a31d77273 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:16.471869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f38a5c7c-b11a-4f05-b5ae-389a31d77273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:16.481599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:16.638422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75e49c-baf2-4781-ba91-d17f5c050df9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:16.661602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75e49c-baf2-4781-ba91-d17f5c050df9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:16.668864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2b2df-31c5-404a-92f9-0972ec8c8e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:16.669495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2b2df-31c5-404a-92f9-0972ec8c8e04 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:16.669778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2b2df-31c5-404a-92f9-0972ec8c8e04 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:16.670052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2b2df-31c5-404a-92f9-0972ec8c8e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:16.678940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2b2df-31c5-404a-92f9-0972ec8c8e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:16.680564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2b2df-31c5-404a-92f9-0972ec8c8e04 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:16.690257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 711/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:17.493702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957a63-25b3-41dc-ba45-7656b732e39c None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:17.494605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957a63-25b3-41dc-ba45-7656b732e39c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:17.494605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957a63-25b3-41dc-ba45-7656b732e39c None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:17.497309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:17.504303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d7a92-4fc2-4489-8aae-190cadd4a42e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:17.504748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d7a92-4fc2-4489-8aae-190cadd4a42e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:17.504907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d7a92-4fc2-4489-8aae-190cadd4a42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:17.528339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-659d7a92-4fc2-4489-8aae-190cadd4a42e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:17.529478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-659d7a92-4fc2-4489-8aae-190cadd4a42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:17.538122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:17.693606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6957a63-25b3-41dc-ba45-7656b732e39c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:17.712275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6957a63-25b3-41dc-ba45-7656b732e39c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:17.718955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d378ae3-486b-4d3b-b996-f4f3855e4b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:17.719863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d378ae3-486b-4d3b-b996-f4f3855e4b7d service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:17.720198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d378ae3-486b-4d3b-b996-f4f3855e4b7d service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:17.720586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d378ae3-486b-4d3b-b996-f4f3855e4b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:17.733978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d378ae3-486b-4d3b-b996-f4f3855e4b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:17.735641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d378ae3-486b-4d3b-b996-f4f3855e4b7d service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:17.745661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 722/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:18.543196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67105f45-6a8b-46ae-9d28-510b9e17ef6e None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:18.544058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67105f45-6a8b-46ae-9d28-510b9e17ef6e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:18.544058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67105f45-6a8b-46ae-9d28-510b9e17ef6e None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:18.546158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:18.551491 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a944468-ed4c-46cd-a7e6-463dd10fb660 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:18.551894 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a944468-ed4c-46cd-a7e6-463dd10fb660 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:18.552085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a944468-ed4c-46cd-a7e6-463dd10fb660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:18.572115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5a944468-ed4c-46cd-a7e6-463dd10fb660 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:18.573334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5a944468-ed4c-46cd-a7e6-463dd10fb660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:18.583383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:18.736784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67105f45-6a8b-46ae-9d28-510b9e17ef6e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:18.758883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67105f45-6a8b-46ae-9d28-510b9e17ef6e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:18.765548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8536b6e-326d-4a0b-b0cb-aaf4f9001a31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:18.766194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8536b6e-326d-4a0b-b0cb-aaf4f9001a31 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:18.766474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8536b6e-326d-4a0b-b0cb-aaf4f9001a31 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:18.766782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8536b6e-326d-4a0b-b0cb-aaf4f9001a31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:18.775493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8536b6e-326d-4a0b-b0cb-aaf4f9001a31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:18.777120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8536b6e-326d-4a0b-b0cb-aaf4f9001a31 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:18.786950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 714/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:19.586093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24f3f-88d2-4d8b-a426-2235c85cf3d0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:19.586885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24f3f-88d2-4d8b-a426-2235c85cf3d0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:19.586885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee24f3f-88d2-4d8b-a426-2235c85cf3d0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:19.589064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:44:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:19.595752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258d20-c373-4f1f-8519-5ebd9201c19b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:19.596226 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258d20-c373-4f1f-8519-5ebd9201c19b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:19.596367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258d20-c373-4f1f-8519-5ebd9201c19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:19.615544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d7258d20-c373-4f1f-8519-5ebd9201c19b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:19.616772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d7258d20-c373-4f1f-8519-5ebd9201c19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:19.625645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:44:19.780712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee24f3f-88d2-4d8b-a426-2235c85cf3d0 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:19.804415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee24f3f-88d2-4d8b-a426-2235c85cf3d0 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:19.812194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad9eeb-d40e-42ed-a8c0-ac90808adeb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:19.812838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad9eeb-d40e-42ed-a8c0-ac90808adeb6 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:19.813120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad9eeb-d40e-42ed-a8c0-ac90808adeb6 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:19.813387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad9eeb-d40e-42ed-a8c0-ac90808adeb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:19.822479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad9eeb-d40e-42ed-a8c0-ac90808adeb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:19.824110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad9eeb-d40e-42ed-a8c0-ac90808adeb6 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:19.833936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 725/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:21.731279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4acaa7-7c06-47f3-a3ac-cdea87dfb3d4 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:21.731893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4acaa7-7c06-47f3-a3ac-cdea87dfb3d4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:21.731893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4acaa7-7c06-47f3-a3ac-cdea87dfb3d4 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:21.733864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:21.740495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5afcbf-f94a-4859-ad0f-4ddc27ca7a9b None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:21.740926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5afcbf-f94a-4859-ad0f-4ddc27ca7a9b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:21.741067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5afcbf-f94a-4859-ad0f-4ddc27ca7a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:21.763011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6b5afcbf-f94a-4859-ad0f-4ddc27ca7a9b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:21.764264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6b5afcbf-f94a-4859-ad0f-4ddc27ca7a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:21.773314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:21.814116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4acaa7-7c06-47f3-a3ac-cdea87dfb3d4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:21.827061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4acaa7-7c06-47f3-a3ac-cdea87dfb3d4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:21.831777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9badd-a19e-4026-bf42-a5985075d6ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:21.832435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9badd-a19e-4026-bf42-a5985075d6ae service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:21.832711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9badd-a19e-4026-bf42-a5985075d6ae service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:21.832981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9badd-a19e-4026-bf42-a5985075d6ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:21.842938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9badd-a19e-4026-bf42-a5985075d6ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:21.844400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9badd-a19e-4026-bf42-a5985075d6ae service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:21.853955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 717/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:24.046779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf193c-976b-4c74-972d-cb3b2718b90b None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:24.047383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf193c-976b-4c74-972d-cb3b2718b90b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:24.047383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf193c-976b-4c74-972d-cb3b2718b90b None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:24.048781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:24.055614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1be3-8f0b-4a12-821c-03424932bedb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:24.056122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1be3-8f0b-4a12-821c-03424932bedb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:24.056299 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc1be3-8f0b-4a12-821c-03424932bedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:24.080631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6cfc1be3-8f0b-4a12-821c-03424932bedb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:24.082309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6cfc1be3-8f0b-4a12-821c-03424932bedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:24.090688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:24.133203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf193c-976b-4c74-972d-cb3b2718b90b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:24.146255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf193c-976b-4c74-972d-cb3b2718b90b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:24.151014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecce633-4628-4565-8701-802dc793e61c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:24.151677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecce633-4628-4565-8701-802dc793e61c service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:24.151959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecce633-4628-4565-8701-802dc793e61c service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:24.152264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecce633-4628-4565-8701-802dc793e61c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:24.162576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecce633-4628-4565-8701-802dc793e61c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:24.164024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecce633-4628-4565-8701-802dc793e61c service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:24.176579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 728/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:44:26.524775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350fbe-8013-4b7c-b308-34833913fb57 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:26.525311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350fbe-8013-4b7c-b308-34833913fb57 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:26.525311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350fbe-8013-4b7c-b308-34833913fb57 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:26.526624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:26.532309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c766d8a-d427-4260-8fd8-c20d9f442245 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:26.532760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c766d8a-d427-4260-8fd8-c20d9f442245 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:26.532860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c766d8a-d427-4260-8fd8-c20d9f442245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:26.553265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-2c766d8a-d427-4260-8fd8-c20d9f442245 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:26.554482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2c766d8a-d427-4260-8fd8-c20d9f442245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:26.564173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:26.606712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350fbe-8013-4b7c-b308-34833913fb57 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:26.625037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350fbe-8013-4b7c-b308-34833913fb57 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:26.629385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40a27b-4bee-4fa1-af6d-e6bdbeec2799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:26.630022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40a27b-4bee-4fa1-af6d-e6bdbeec2799 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:26.630298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40a27b-4bee-4fa1-af6d-e6bdbeec2799 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:26.630565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40a27b-4bee-4fa1-af6d-e6bdbeec2799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:26.640623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40a27b-4bee-4fa1-af6d-e6bdbeec2799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:26.642049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40a27b-4bee-4fa1-af6d-e6bdbeec2799 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:26.651642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 720/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:28.861111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13740f-d83f-405e-a9ff-6db293896761 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:28.861757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13740f-d83f-405e-a9ff-6db293896761 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:28.861757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13740f-d83f-405e-a9ff-6db293896761 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:28.863095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:28.869609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f540f6-6ce8-4068-ac69-7bca7bf6f7f2 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:28.870688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f540f6-6ce8-4068-ac69-7bca7bf6f7f2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:28.870842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f540f6-6ce8-4068-ac69-7bca7bf6f7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:28.892580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d1f540f6-6ce8-4068-ac69-7bca7bf6f7f2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:28.894317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d1f540f6-6ce8-4068-ac69-7bca7bf6f7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:28.906577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:28.949763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13740f-d83f-405e-a9ff-6db293896761 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:28.962269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13740f-d83f-405e-a9ff-6db293896761 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:28.966675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8300b60-5e12-43f0-bce2-ffdc87d7bbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:28.967309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300b60-5e12-43f0-bce2-ffdc87d7bbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:28.967591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300b60-5e12-43f0-bce2-ffdc87d7bbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:28.967886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8300b60-5e12-43f0-bce2-ffdc87d7bbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:28.978199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8300b60-5e12-43f0-bce2-ffdc87d7bbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:28.979638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8300b60-5e12-43f0-bce2-ffdc87d7bbd3 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:28.989303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 731/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:31.250343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8b57f-0c2c-4fc2-84f5-491023ef7394 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:31.251280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8b57f-0c2c-4fc2-84f5-491023ef7394 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:31.251280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8b57f-0c2c-4fc2-84f5-491023ef7394 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:31.253613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:31.265024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3f9e-91e3-4350-910e-dbf63306cc3a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:31.265811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3f9e-91e3-4350-910e-dbf63306cc3a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:31.266717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b3f9e-91e3-4350-910e-dbf63306cc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:31.287028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0b2b3f9e-91e3-4350-910e-dbf63306cc3a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:31.288281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0b2b3f9e-91e3-4350-910e-dbf63306cc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:31.297991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:31.339895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8b57f-0c2c-4fc2-84f5-491023ef7394 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:31.356420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8b57f-0c2c-4fc2-84f5-491023ef7394 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:31.361186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e2472-78fe-419d-97ca-baf993fae4d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:31.361819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2472-78fe-419d-97ca-baf993fae4d2 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:31.362100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2472-78fe-419d-97ca-baf993fae4d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:31.362371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2472-78fe-419d-97ca-baf993fae4d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:31.372285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e2472-78fe-419d-97ca-baf993fae4d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:31.373727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e2472-78fe-419d-97ca-baf993fae4d2 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:31.383611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 723/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:33.433298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108d7a-0925-47bb-8566-aa5488a8dc74 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:33.433850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108d7a-0925-47bb-8566-aa5488a8dc74 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:33.433850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108d7a-0925-47bb-8566-aa5488a8dc74 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:33.435212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:33.440571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b88d0-c240-4ff2-b64b-a0b0601d889f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:33.441070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b88d0-c240-4ff2-b64b-a0b0601d889f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:33.441070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b88d0-c240-4ff2-b64b-a0b0601d889f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:33.460033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f15b88d0-c240-4ff2-b64b-a0b0601d889f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:33.461229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f15b88d0-c240-4ff2-b64b-a0b0601d889f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:33.469672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:33.507907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33108d7a-0925-47bb-8566-aa5488a8dc74 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:33.519356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33108d7a-0925-47bb-8566-aa5488a8dc74 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:33.523253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4834bae-2561-4095-983c-5afdb38f476c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:33.523741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4834bae-2561-4095-983c-5afdb38f476c service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:33.524022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4834bae-2561-4095-983c-5afdb38f476c service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:33.524316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4834bae-2561-4095-983c-5afdb38f476c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:33.533799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4834bae-2561-4095-983c-5afdb38f476c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:33.535240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4834bae-2561-4095-983c-5afdb38f476c service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:33.544676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 734/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:35.621183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb16f4-2fa1-46ec-ba3e-d76c29595452 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:35.622091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb16f4-2fa1-46ec-ba3e-d76c29595452 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:35.622091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb16f4-2fa1-46ec-ba3e-d76c29595452 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:35.624501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:35.629765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42984d-91ea-48a2-a9a0-c366202e94fe None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:35.630263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42984d-91ea-48a2-a9a0-c366202e94fe None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:35.630263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42984d-91ea-48a2-a9a0-c366202e94fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:35.648906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0f42984d-91ea-48a2-a9a0-c366202e94fe None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:35.650090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0f42984d-91ea-48a2-a9a0-c366202e94fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:35.658598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:35.696430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb16f4-2fa1-46ec-ba3e-d76c29595452 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:35.710820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb16f4-2fa1-46ec-ba3e-d76c29595452 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:35.715037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3494e23-8573-4fbf-bc89-35dca9ee1351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:35.715674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3494e23-8573-4fbf-bc89-35dca9ee1351 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:35.715959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3494e23-8573-4fbf-bc89-35dca9ee1351 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:35.716274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3494e23-8573-4fbf-bc89-35dca9ee1351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:35.726486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3494e23-8573-4fbf-bc89-35dca9ee1351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:35.727919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3494e23-8573-4fbf-bc89-35dca9ee1351 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:35.738034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 726/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:37.783735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab735-8f4f-423e-8b3a-a2c3ac66aee1 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:37.784332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab735-8f4f-423e-8b3a-a2c3ac66aee1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:37.784332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ab735-8f4f-423e-8b3a-a2c3ac66aee1 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:37.785722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:37.791209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11fd8b-7fa5-4181-9712-01c1989ce0fc None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:37.791653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11fd8b-7fa5-4181-9712-01c1989ce0fc None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:37.791749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11fd8b-7fa5-4181-9712-01c1989ce0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:37.798717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=79 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:44:37.812665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-eb11fd8b-7fa5-4181-9712-01c1989ce0fc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:37.813917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-eb11fd8b-7fa5-4181-9712-01c1989ce0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:37.822755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:37.860224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ab735-8f4f-423e-8b3a-a2c3ac66aee1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:37.866432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=73 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:44:37.873446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ab735-8f4f-423e-8b3a-a2c3ac66aee1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:37.877788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc506b17-442a-4097-83e5-4c94b3e6520a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:37.878431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc506b17-442a-4097-83e5-4c94b3e6520a service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:37.878715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc506b17-442a-4097-83e5-4c94b3e6520a service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:37.878985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc506b17-442a-4097-83e5-4c94b3e6520a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:37.888507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc506b17-442a-4097-83e5-4c94b3e6520a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:37.890011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc506b17-442a-4097-83e5-4c94b3e6520a service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:37.899777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 737/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:39.951613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1066be1-e62a-4453-8006-0255a53cafa1 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:39.952262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1066be1-e62a-4453-8006-0255a53cafa1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:39.952262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1066be1-e62a-4453-8006-0255a53cafa1 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:39.953655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:39.960154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d720c59-bb2b-4216-b10b-2854df9de161 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:39.960866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d720c59-bb2b-4216-b10b-2854df9de161 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:39.961018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d720c59-bb2b-4216-b10b-2854df9de161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:39.982919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3d720c59-bb2b-4216-b10b-2854df9de161 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:39.984180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3d720c59-bb2b-4216-b10b-2854df9de161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:39.993993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:44:40.030623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1066be1-e62a-4453-8006-0255a53cafa1 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:40.046477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1066be1-e62a-4453-8006-0255a53cafa1 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:40.053287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cccad-5302-46cb-a711-16c007850552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:40.054187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cccad-5302-46cb-a711-16c007850552 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:40.054628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cccad-5302-46cb-a711-16c007850552 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:40.054968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cccad-5302-46cb-a711-16c007850552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:40.068029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cccad-5302-46cb-a711-16c007850552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:40.069472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cccad-5302-46cb-a711-16c007850552 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:40.079376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 729/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:44:42.101150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c5505-66cc-4252-bda9-9670f5b75582 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:42.102059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c5505-66cc-4252-bda9-9670f5b75582 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:42.102059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c5505-66cc-4252-bda9-9670f5b75582 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:42.104393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:44:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:44:42.109996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53db3b-1929-4bd2-97fc-bf1a45a4d55f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:42.110452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53db3b-1929-4bd2-97fc-bf1a45a4d55f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:42.110559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53db3b-1929-4bd2-97fc-bf1a45a4d55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:42.134720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-0f53db3b-1929-4bd2-97fc-bf1a45a4d55f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:44:42.135925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-0f53db3b-1929-4bd2-97fc-bf1a45a4d55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:44:42.144933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:44:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:44:42.184241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c5505-66cc-4252-bda9-9670f5b75582 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:44:42.196443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c5505-66cc-4252-bda9-9670f5b75582 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:44:42.200922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51a46a-ccf3-47ff-953b-a1c1908f8cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:44:42.201556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a46a-ccf3-47ff-953b-a1c1908f8cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:44:42.201839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a46a-ccf3-47ff-953b-a1c1908f8cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:44:42.202114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a46a-ccf3-47ff-953b-a1c1908f8cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:44:42.211737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51a46a-ccf3-47ff-953b-a1c1908f8cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:44:42.213198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51a46a-ccf3-47ff-953b-a1c1908f8cf0 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:44:42.222845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 740/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:44:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:44:52.129308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=11 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:44:52.218649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=14 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:45:02.067157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6db67-0304-4e67-89d1-e6fbb33a9c4c None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:02.068175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6db67-0304-4e67-89d1-e6fbb33a9c4c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:02.068456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6db67-0304-4e67-89d1-e6fbb33a9c4c None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:02.070837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 731/1471] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:45:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:02.077448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259b6be-d63e-4777-a32a-6c1c53847654 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:02.078232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259b6be-d63e-4777-a32a-6c1c53847654 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:02.078547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259b6be-d63e-4777-a32a-6c1c53847654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:02.097168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7259b6be-d63e-4777-a32a-6c1c53847654 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:02.098401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7259b6be-d63e-4777-a32a-6c1c53847654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:02.107354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 741/1472] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:45:02.155220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cd4f2-3f1f-4b29-b070-a5a322e82528 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:02.155508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cd4f2-3f1f-4b29-b070-a5a322e82528 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:02.155786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cd4f2-3f1f-4b29-b070-a5a322e82528 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:02.157329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 732/1473] 10.222.0.50 () {54 vars in 962 bytes} [Thu Jun 30 15:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:02.162446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb818a-8c65-4cc7-9946-c9e3b6e03e04 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:02.162736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb818a-8c65-4cc7-9946-c9e3b6e03e04 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:02.163014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb818a-8c65-4cc7-9946-c9e3b6e03e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:02.181565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a1cb818a-8c65-4cc7-9946-c9e3b6e03e04 None None] MFA Rules not processed for user `e4cb4d5213c647459e1c21c311a4a305`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:02.182764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a1cb818a-8c65-4cc7-9946-c9e3b6e03e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:02.190091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 742/1474] 10.222.0.50 () {58 vars in 1059 bytes} [Thu Jun 30 15:45:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:45:02.197497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cd4f2-3f1f-4b29-b070-a5a322e82528 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:45:02.211702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cd4f2-3f1f-4b29-b070-a5a322e82528 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:45:02.215665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa51c2b-a248-469c-9eaa-50a177eb6ac0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:45:02.216377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51c2b-a248-469c-9eaa-50a177eb6ac0 service glance] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:02.216568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51c2b-a248-469c-9eaa-50a177eb6ac0 service glance] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:02.216860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa51c2b-a248-469c-9eaa-50a177eb6ac0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:02.226512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51c2b-a248-469c-9eaa-50a177eb6ac0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:45:02.227945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa51c2b-a248-469c-9eaa-50a177eb6ac0 service glance] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:45:02.237486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 733/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:45:04.355860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258238d-6403-4d6a-a80d-82609d697a77 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:04.356520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258238d-6403-4d6a-a80d-82609d697a77 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:04.356520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258238d-6403-4d6a-a80d-82609d697a77 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:04.357890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 743/1476] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:04.364294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd47ac-6240-46e4-a5c9-99e011a795d2 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:04.364984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd47ac-6240-46e4-a5c9-99e011a795d2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:04.365168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd47ac-6240-46e4-a5c9-99e011a795d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:04.394200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-b0cd47ac-6240-46e4-a5c9-99e011a795d2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:04.395420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b0cd47ac-6240-46e4-a5c9-99e011a795d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:04.404457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 734/1477] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:45:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:45:04.428020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258238d-6403-4d6a-a80d-82609d697a77 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:45:04.448380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258238d-6403-4d6a-a80d-82609d697a77 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:45:04.453371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc9bd6-2360-4469-b0fe-bd1385aff6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:45:04.454068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc9bd6-2360-4469-b0fe-bd1385aff6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:04.454359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc9bd6-2360-4469-b0fe-bd1385aff6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:04.454621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc9bd6-2360-4469-b0fe-bd1385aff6e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:04.455385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc9bd6-2360-4469-b0fe-bd1385aff6e6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:45:04.457498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-8abc9bd6-2360-4469-b0fe-bd1385aff6e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:45:04.457498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:45:04.457498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:45:04.458638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc9bd6-2360-4469-b0fe-bd1385aff6e6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:45:04.478373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 744/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Thu Jun 30 15:45:04 2022] GET /identity/v3/projects => generated 2553 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:45:06.532400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e9b9-6252-4578-a56d-6da078373747 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:06.532970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e9b9-6252-4578-a56d-6da078373747 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:06.532970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e9b9-6252-4578-a56d-6da078373747 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:06.534249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 735/1479] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:06.540635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4057c-0cbb-4e26-ba87-1c4accff6518 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:06.541086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4057c-0cbb-4e26-ba87-1c4accff6518 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:06.541199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4057c-0cbb-4e26-ba87-1c4accff6518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:06.566736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4ce4057c-0cbb-4e26-ba87-1c4accff6518 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:06.567976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4ce4057c-0cbb-4e26-ba87-1c4accff6518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:06.579317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 745/1480] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:45:06.638344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72e9b9-6252-4578-a56d-6da078373747 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:45:06.653622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72e9b9-6252-4578-a56d-6da078373747 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:45:06.659358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063a3d6-3d55-4987-8b65-80983c7be8f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:45:06.659981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063a3d6-3d55-4987-8b65-80983c7be8f8 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:06.660297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063a3d6-3d55-4987-8b65-80983c7be8f8 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:06.660565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063a3d6-3d55-4987-8b65-80983c7be8f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:06.668584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063a3d6-3d55-4987-8b65-80983c7be8f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:45:06.670153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063a3d6-3d55-4987-8b65-80983c7be8f8 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:45:06.679437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 736/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:45:08.922375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dc745-a5e7-4fd3-8428-81f82eb5a198 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:08.923317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dc745-a5e7-4fd3-8428-81f82eb5a198 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:08.923486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dc745-a5e7-4fd3-8428-81f82eb5a198 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:08.925728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 746/1482] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:45:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:08.934649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b59eea-f4d7-44bd-bf13-689ed0be9ec7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:08.935343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b59eea-f4d7-44bd-bf13-689ed0be9ec7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:08.935522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b59eea-f4d7-44bd-bf13-689ed0be9ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:08.960649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e1b59eea-f4d7-44bd-bf13-689ed0be9ec7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:08.961849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e1b59eea-f4d7-44bd-bf13-689ed0be9ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:08.970447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 737/1483] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:45:09.011327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dc745-a5e7-4fd3-8428-81f82eb5a198 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:45:09.030648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dc745-a5e7-4fd3-8428-81f82eb5a198 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:45:09.035808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c872410-a469-40be-9d20-febcf4c92129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:45:09.036476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c872410-a469-40be-9d20-febcf4c92129 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:09.036757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c872410-a469-40be-9d20-febcf4c92129 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:09.037027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c872410-a469-40be-9d20-febcf4c92129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:09.046342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c872410-a469-40be-9d20-febcf4c92129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:45:09.047778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c872410-a469-40be-9d20-febcf4c92129 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:45:09.057440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 747/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:45:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:45:11.163283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f27a28-7892-4538-9808-162272fd3904 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:11.163875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f27a28-7892-4538-9808-162272fd3904 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:11.163875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f27a28-7892-4538-9808-162272fd3904 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:11.165260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 738/1485] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:11.171807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7c7ad-67ea-4711-9aab-44cc4c1c6df5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:11.172568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7c7ad-67ea-4711-9aab-44cc4c1c6df5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:11.172889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7c7ad-67ea-4711-9aab-44cc4c1c6df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:11.193622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-44a7c7ad-67ea-4711-9aab-44cc4c1c6df5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:11.194840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-44a7c7ad-67ea-4711-9aab-44cc4c1c6df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:11.205303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 748/1486] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:45:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:45:11.246747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f27a28-7892-4538-9808-162272fd3904 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:45:11.260583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f27a28-7892-4538-9808-162272fd3904 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:45:11.264628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bb643-5a8e-40d6-b2e9-a7af9f926739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:45:11.265302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bb643-5a8e-40d6-b2e9-a7af9f926739 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:11.265581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bb643-5a8e-40d6-b2e9-a7af9f926739 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:11.265827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bb643-5a8e-40d6-b2e9-a7af9f926739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:11.275630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bb643-5a8e-40d6-b2e9-a7af9f926739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:45:11.277127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2bb643-5a8e-40d6-b2e9-a7af9f926739 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:45:11.287293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 739/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:45:13.415489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459188b1-73d1-43d9-a7f2-546bb4f9e3da None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:13.416078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459188b1-73d1-43d9-a7f2-546bb4f9e3da None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:13.416078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459188b1-73d1-43d9-a7f2-546bb4f9e3da None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:13.418049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 749/1488] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:13.423658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6be02-406c-40ce-9769-2d218fe0ad27 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:13.424136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6be02-406c-40ce-9769-2d218fe0ad27 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:13.424136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6be02-406c-40ce-9769-2d218fe0ad27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:13.442723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-dde6be02-406c-40ce-9769-2d218fe0ad27 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:13.443912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-dde6be02-406c-40ce-9769-2d218fe0ad27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:13.452275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 740/1489] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:45:13.496663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459188b1-73d1-43d9-a7f2-546bb4f9e3da None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:45:13.511049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459188b1-73d1-43d9-a7f2-546bb4f9e3da None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:45:13.515413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d230d-e6e3-465f-b21b-37ee1cdc03ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:45:13.516094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d230d-e6e3-465f-b21b-37ee1cdc03ad service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:13.516389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d230d-e6e3-465f-b21b-37ee1cdc03ad service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:13.516655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d230d-e6e3-465f-b21b-37ee1cdc03ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:13.526262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d230d-e6e3-465f-b21b-37ee1cdc03ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:45:13.527656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d230d-e6e3-465f-b21b-37ee1cdc03ad service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:45:13.537439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 750/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:45:15.772787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baefe23-e327-4426-b551-abdbeb240168 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:15.773377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baefe23-e327-4426-b551-abdbeb240168 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:15.773377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baefe23-e327-4426-b551-abdbeb240168 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:15.774747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 741/1491] 10.222.0.50 () {54 vars in 926 bytes} [Thu Jun 30 15:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:45:15.781226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786de51-5fcc-4d23-b756-f458d46c5843 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:15.781973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786de51-5fcc-4d23-b756-f458d46c5843 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:15.782211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6786de51-5fcc-4d23-b756-f458d46c5843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:15.802182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6786de51-5fcc-4d23-b756-f458d46c5843 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:45:15.803628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6786de51-5fcc-4d23-b756-f458d46c5843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:45:15.813006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 751/1492] 10.222.0.50 () {58 vars in 1023 bytes} [Thu Jun 30 15:45:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:45:15.856547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baefe23-e327-4426-b551-abdbeb240168 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:45:15.876571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baefe23-e327-4426-b551-abdbeb240168 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:45:15.881010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb589-1206-4f55-818b-178376515b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:45:15.881678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb589-1206-4f55-818b-178376515b8b service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:45:15.881962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb589-1206-4f55-818b-178376515b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:45:15.882235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb589-1206-4f55-818b-178376515b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:45:15.892135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb589-1206-4f55-818b-178376515b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:45:15.893587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb589-1206-4f55-818b-178376515b8b service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:45:15.903504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 742/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:45:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:45:25.795468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=33 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:45:25.899133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=33 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:46:44.160291 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b184a-8a45-441e-9437-53f66964afbb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.161052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b184a-8a45-441e-9437-53f66964afbb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.161052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b184a-8a45-441e-9437-53f66964afbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.180883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-089b184a-8a45-441e-9437-53f66964afbb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:46:44.182059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-089b184a-8a45-441e-9437-53f66964afbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:46:44.191366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 752/1494] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:46:44.200342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a43f8-47ba-4931-b663-48a5378566ea None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.200860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a43f8-47ba-4931-b663-48a5378566ea None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.200860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a43f8-47ba-4931-b663-48a5378566ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.222196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-692a43f8-47ba-4931-b663-48a5378566ea None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:46:44.223426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-692a43f8-47ba-4931-b663-48a5378566ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:46:44.231672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 743/1495] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:46:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:46:44.236367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b184a-8a45-441e-9437-53f66964afbb None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.247017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b184a-8a45-441e-9437-53f66964afbb None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.252151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.252794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.253068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.253334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.254054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:46:44.254405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.255418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:44.255418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:46:44.255418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:46:44.256312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:44.273686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 753/1496] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:46:44.281980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a43f8-47ba-4931-b663-48a5378566ea None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.291757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a43f8-47ba-4931-b663-48a5378566ea None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.296931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.297576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.297853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.298120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.298951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.300899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING py.warnings [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:44.300899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: warnings.warn(msg) Jun 30 15:46:44.300899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: Jun 30 15:46:44.302106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:44.346087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 744/1497] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:46:44 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:46:44.350582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.360669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f739a6-e4ce-4891-9476-68a5b5e37758 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.417222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.417827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.418105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.418378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.419075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.420159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:44.420159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:46:44.420159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:46:44.421349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:44.445631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 754/1498] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:46:44 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:46:44.449891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.460321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4a051-e9be-4bc4-8669-95efcf78b491 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.465006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.465662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.465949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.466230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.468874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.471772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING py.warnings [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:44.471772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: warnings.warn(msg) Jun 30 15:46:44.471772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: Jun 30 15:46:44.472706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:44.485868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 745/1499] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:46:44.490465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.501843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cd90d-995d-442f-84c5-2c7645e428d1 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.507183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94922a9-8bd2-4706-be0d-72080cc3c640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.507836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94922a9-8bd2-4706-be0d-72080cc3c640 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.508156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94922a9-8bd2-4706-be0d-72080cc3c640 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.508435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94922a9-8bd2-4706-be0d-72080cc3c640 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.509172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94922a9-8bd2-4706-be0d-72080cc3c640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.510195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-d94922a9-8bd2-4706-be0d-72080cc3c640 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:44.510195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:46:44.510195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:46:44.511086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94922a9-8bd2-4706-be0d-72080cc3c640 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:44.524689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 755/1500] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:46:44.529496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.542014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b6672f-8d05-4021-a721-885fca8d10df admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.547633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43954308-4446-4fdb-8c43-58d19065db44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.548553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954308-4446-4fdb-8c43-58d19065db44 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.548835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954308-4446-4fdb-8c43-58d19065db44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.549108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43954308-4446-4fdb-8c43-58d19065db44 admin admin] PATH_INFO: `/v3/projects/a7f75577be3b45a89778b6a1ea5769bf/users/b475b6bb350f43599c59507fe99f30f7/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.564252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43954308-4446-4fdb-8c43-58d19065db44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f75577be3b45a89778b6a1ea5769bf, user_id=b475b6bb350f43599c59507fe99f30f7, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.565284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING py.warnings [None req-43954308-4446-4fdb-8c43-58d19065db44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:44.565284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: warnings.warn(msg) Jun 30 15:46:44.565284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: Jun 30 15:46:44.567195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43954308-4446-4fdb-8c43-58d19065db44 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:44.582143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 746/1501] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:46:44 2022] PUT /identity/v3/projects/a7f75577be3b45a89778b6a1ea5769bf/users/b475b6bb350f43599c59507fe99f30f7/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:46:44.594816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e732-27d7-45ca-bed7-1df76155d381 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.595102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e732-27d7-45ca-bed7-1df76155d381 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.595372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446e732-27d7-45ca-bed7-1df76155d381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.623461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5446e732-27d7-45ca-bed7-1df76155d381 None None] MFA Rules not processed for user `b475b6bb350f43599c59507fe99f30f7`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:46:44.624705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5446e732-27d7-45ca-bed7-1df76155d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:46:44.693473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 756/1502] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:46:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:46:44.709041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197e171-77d1-4cfa-a1b7-e6e5904846e4 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.709331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197e171-77d1-4cfa-a1b7-e6e5904846e4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.709695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197e171-77d1-4cfa-a1b7-e6e5904846e4 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.711084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 747/1503] 10.222.0.50 () {54 vars in 963 bytes} [Thu Jun 30 15:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:46:44.718435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dda77-5508-4a12-90a5-9b8fcf759755 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.718795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dda77-5508-4a12-90a5-9b8fcf759755 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.719153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3dda77-5508-4a12-90a5-9b8fcf759755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.743436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7f3dda77-5508-4a12-90a5-9b8fcf759755 None None] MFA Rules not processed for user `7c9cdda3d7884298a791f90523f3ab91`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:46:44.744654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7f3dda77-5508-4a12-90a5-9b8fcf759755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:46:44.788544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:46:44.797218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 757/1504] 10.222.0.50 () {58 vars in 1060 bytes} [Thu Jun 30 15:46:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:46:44.804927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197e171-77d1-4cfa-a1b7-e6e5904846e4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.816504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197e171-77d1-4cfa-a1b7-e6e5904846e4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.820824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.821458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.821739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.822014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.830594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.832056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:44.841159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 748/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:46:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:46:44.884276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05bc88-170b-4b72-8228-2a7be204efee None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.884564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05bc88-170b-4b72-8228-2a7be204efee None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.884901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05bc88-170b-4b72-8228-2a7be204efee None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.886188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 758/1506] 10.222.0.50 () {52 vars in 829 bytes} [Thu Jun 30 15:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:46:44.890475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:44.955448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad6aa8-b887-42e6-bc6f-8efe68a9e093 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:44.961048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:44.961952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:44.962234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:44.962507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] PATH_INFO: `/v3/users/b475b6bb350f43599c59507fe99f30f7` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:44.963807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b475b6bb350f43599c59507fe99f30f7)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:44.965004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING py.warnings [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:44.965004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: warnings.warn(msg) Jun 30 15:46:44.965004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: Jun 30 15:46:44.966211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:45.039798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b475b6bb350f43599c59507fe99f30f7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.051376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b475b6bb350f43599c59507fe99f30f7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.063916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b475b6bb350f43599c59507fe99f30f7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.064667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b475b6bb350f43599c59507fe99f30f7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.065396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b475b6bb350f43599c59507fe99f30f7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.082235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:46:45.091285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 749/1507] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:46:44 2022] DELETE /identity/v3/users/b475b6bb350f43599c59507fe99f30f7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:46:45.095733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:45.096075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:45.096352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:45.114503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:46:45.115763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:46:45.176609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 759/1508] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:46:45.186449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:45.188213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:46:45.251415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e323d9a-a5f0-4b2a-a159-b26566828e04 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:45.256812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a32393-3aff-44c8-bca8-cf7274c210e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:45.257468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a32393-3aff-44c8-bca8-cf7274c210e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:45.257730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a32393-3aff-44c8-bca8-cf7274c210e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:45.258026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a32393-3aff-44c8-bca8-cf7274c210e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:45.268912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a32393-3aff-44c8-bca8-cf7274c210e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:45.270430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a32393-3aff-44c8-bca8-cf7274c210e5 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:45.280793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 750/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:46:45.618357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:46:45.619867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:46:45.632335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100427b-b6da-48ae-b895-363121d2dabc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:46:45.637468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:46:45.638453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:46:45.638730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:46:45.642939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] PATH_INFO: `/v3/projects/a7f75577be3b45a89778b6a1ea5769bf` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:46:45.642939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f75577be3b45a89778b6a1ea5769bf)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:46:45.642939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:46:45.642939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:46:45.642939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:46:45.642939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:46:45.710832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f75577be3b45a89778b6a1ea5769bf', 'request_id': 'req-5521acab-3c85-427d-abae-8a23f2c6acf6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.722992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f75577be3b45a89778b6a1ea5769bf', 'request_id': 'req-5521acab-3c85-427d-abae-8a23f2c6acf6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.736673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f75577be3b45a89778b6a1ea5769bf', 'request_id': 'req-5521acab-3c85-427d-abae-8a23f2c6acf6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.747890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] The token cache is being invalidate because project a7f75577be3b45a89778b6a1ea5769bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:46:45.748195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.748863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5521acab-3c85-427d-abae-8a23f2c6acf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:46:45.750521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 760/1510] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:46:45 2022] DELETE /identity/v3/projects/a7f75577be3b45a89778b6a1ea5769bf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:46:55.276050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:46:55.744508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:48:28.784063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872e17f-4b2c-4df9-bd9c-ec7f7fa14b4d None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:28.784675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872e17f-4b2c-4df9-bd9c-ec7f7fa14b4d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:28.784675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b872e17f-4b2c-4df9-bd9c-ec7f7fa14b4d None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:28.786382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:48:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:28.793583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725dbc1-f7e2-4489-adc9-486e43d03334 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:28.794071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725dbc1-f7e2-4489-adc9-486e43d03334 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:28.794177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725dbc1-f7e2-4489-adc9-486e43d03334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:28.815863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b725dbc1-f7e2-4489-adc9-486e43d03334 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:28.817106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b725dbc1-f7e2-4489-adc9-486e43d03334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:28.877377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:29.028062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872e17f-4b2c-4df9-bd9c-ec7f7fa14b4d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:29.030588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b872e17f-4b2c-4df9-bd9c-ec7f7fa14b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:29.106646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b872e17f-4b2c-4df9-bd9c-ec7f7fa14b4d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:29.112490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d31b1-c94e-461d-a5a1-353065cce016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:29.113134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d31b1-c94e-461d-a5a1-353065cce016 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:29.113420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d31b1-c94e-461d-a5a1-353065cce016 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:29.113697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d31b1-c94e-461d-a5a1-353065cce016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:29.122310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d31b1-c94e-461d-a5a1-353065cce016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:29.124024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d31b1-c94e-461d-a5a1-353065cce016 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:29.133522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 752/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:29.965814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a10bb8-b38c-4eab-9992-b1abd9750ef5 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:29.966420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a10bb8-b38c-4eab-9992-b1abd9750ef5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:29.966528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a10bb8-b38c-4eab-9992-b1abd9750ef5 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:29.967979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:29.975797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9eba-1b21-4422-9ad9-e4e881e650f3 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:29.976390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9eba-1b21-4422-9ad9-e4e881e650f3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:29.976562 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd9eba-1b21-4422-9ad9-e4e881e650f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:30.000344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3acd9eba-1b21-4422-9ad9-e4e881e650f3 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:30.001547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3acd9eba-1b21-4422-9ad9-e4e881e650f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:30.010119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:30.166204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a10bb8-b38c-4eab-9992-b1abd9750ef5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:30.183224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a10bb8-b38c-4eab-9992-b1abd9750ef5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:30.189128 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd2051-3032-4476-a605-2c69390d6a1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:30.189768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd2051-3032-4476-a605-2c69390d6a1d service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:30.190052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd2051-3032-4476-a605-2c69390d6a1d service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:30.190323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd2051-3032-4476-a605-2c69390d6a1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:30.199125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd2051-3032-4476-a605-2c69390d6a1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:30.200849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd2051-3032-4476-a605-2c69390d6a1d service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:30.211115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 763/1516] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:32.806940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361dc7d-fe7e-4402-9d61-7b6952e4aa58 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:32.807557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361dc7d-fe7e-4402-9d61-7b6952e4aa58 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:32.807557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361dc7d-fe7e-4402-9d61-7b6952e4aa58 None None] PATH_INFO: `/v3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:32.809352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 754/1517] 10.222.0.71 () {54 vars in 925 bytes} [Thu Jun 30 15:48:32 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:48:32.817147 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c0be3-62f2-4b82-8adc-bee03a477cc5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:32.817568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c0be3-62f2-4b82-8adc-bee03a477cc5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:32.817683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c0be3-62f2-4b82-8adc-bee03a477cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:32.837262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-446c0be3-62f2-4b82-8adc-bee03a477cc5 None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:32.838471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-446c0be3-62f2-4b82-8adc-bee03a477cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:32.913860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 764/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Thu Jun 30 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:37.007098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21001fc4-a0ec-47fb-904b-9042e5435618 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:37.007513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21001fc4-a0ec-47fb-904b-9042e5435618 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:37.007625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21001fc4-a0ec-47fb-904b-9042e5435618 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:37.009236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 755/1519] 10.222.0.50 () {54 vars in 960 bytes} [Thu Jun 30 15:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:37.015135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1cd5-eff8-4073-932f-9eb58a1abcfb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:37.015631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1cd5-eff8-4073-932f-9eb58a1abcfb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:37.015740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b1cd5-eff8-4073-932f-9eb58a1abcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:37.038976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0c5b1cd5-eff8-4073-932f-9eb58a1abcfb None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:37.040216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0c5b1cd5-eff8-4073-932f-9eb58a1abcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:37.049088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 765/1520] 10.222.0.50 () {58 vars in 1057 bytes} [Thu Jun 30 15:48:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:37.056677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21001fc4-a0ec-47fb-904b-9042e5435618 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:37.072244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21001fc4-a0ec-47fb-904b-9042e5435618 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:37.078171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362affa-408a-49a9-ae26-89300e43928e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:37.078798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362affa-408a-49a9-ae26-89300e43928e service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:37.079087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362affa-408a-49a9-ae26-89300e43928e service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:37.079352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362affa-408a-49a9-ae26-89300e43928e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:37.088795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362affa-408a-49a9-ae26-89300e43928e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:37.090407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362affa-408a-49a9-ae26-89300e43928e service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:37.100511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 756/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:41.449220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c4c28-0a4d-4504-beda-21d3115fcae1 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:41.449864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c4c28-0a4d-4504-beda-21d3115fcae1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:41.449864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c4c28-0a4d-4504-beda-21d3115fcae1 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:41.451182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:41.457207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dd4f0-a17d-4d12-97e5-fc91db02db7a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:41.458019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dd4f0-a17d-4d12-97e5-fc91db02db7a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:41.458019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dd4f0-a17d-4d12-97e5-fc91db02db7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:41.480998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-968dd4f0-a17d-4d12-97e5-fc91db02db7a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:41.482214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-968dd4f0-a17d-4d12-97e5-fc91db02db7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:41.491502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:41.646600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c4c28-0a4d-4504-beda-21d3115fcae1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:41.662277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c4c28-0a4d-4504-beda-21d3115fcae1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:41.668535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738adb2d-d748-456f-9463-74d0e8e62117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:41.669173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738adb2d-d748-456f-9463-74d0e8e62117 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:41.669451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738adb2d-d748-456f-9463-74d0e8e62117 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:41.669708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738adb2d-d748-456f-9463-74d0e8e62117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:41.678526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738adb2d-d748-456f-9463-74d0e8e62117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:41.680182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738adb2d-d748-456f-9463-74d0e8e62117 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:41.689741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 767/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:42.522497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf1125-d520-483b-bdb7-1d9c3197c3c5 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:42.523051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf1125-d520-483b-bdb7-1d9c3197c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:42.523051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf1125-d520-483b-bdb7-1d9c3197c3c5 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:42.524396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 15:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:42.530468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801061f-c91a-4dbf-a5a8-b81d59f45ccd None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:42.531277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801061f-c91a-4dbf-a5a8-b81d59f45ccd None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:42.531427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801061f-c91a-4dbf-a5a8-b81d59f45ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:42.552980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4801061f-c91a-4dbf-a5a8-b81d59f45ccd None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:42.554176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4801061f-c91a-4dbf-a5a8-b81d59f45ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:42.562942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 15:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:42.722072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf1125-d520-483b-bdb7-1d9c3197c3c5 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:42.745663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf1125-d520-483b-bdb7-1d9c3197c3c5 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:42.753743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f245b82-e14a-47e7-a78b-fc1b8a4f6384 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:42.754373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245b82-e14a-47e7-a78b-fc1b8a4f6384 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:42.754649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245b82-e14a-47e7-a78b-fc1b8a4f6384 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:42.754914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f245b82-e14a-47e7-a78b-fc1b8a4f6384 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:42.764460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f245b82-e14a-47e7-a78b-fc1b8a4f6384 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:42.766068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f245b82-e14a-47e7-a78b-fc1b8a4f6384 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:42.776369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 759/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:46.790565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc99cd-f51e-4108-b391-c2f26ad7e673 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:46.791193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc99cd-f51e-4108-b391-c2f26ad7e673 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:46.791193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc99cd-f51e-4108-b391-c2f26ad7e673 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:46.793557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:48:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:46.801494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0160-d712-4135-b1c2-d8b01680db41 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:46.801928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0160-d712-4135-b1c2-d8b01680db41 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:46.802040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add0160-d712-4135-b1c2-d8b01680db41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:46.822624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8add0160-d712-4135-b1c2-d8b01680db41 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:46.823861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8add0160-d712-4135-b1c2-d8b01680db41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:46.832201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:46.868906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc99cd-f51e-4108-b391-c2f26ad7e673 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:46.872254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6ebc99cd-f51e-4108-b391-c2f26ad7e673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:46.939232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc99cd-f51e-4108-b391-c2f26ad7e673 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:46.943677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0badd-9341-4cee-aaa8-a88313f38002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:46.944362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0badd-9341-4cee-aaa8-a88313f38002 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:46.944636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0badd-9341-4cee-aaa8-a88313f38002 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:46.944895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0badd-9341-4cee-aaa8-a88313f38002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:46.955607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0badd-9341-4cee-aaa8-a88313f38002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:46.957105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0badd-9341-4cee-aaa8-a88313f38002 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:46.962433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:48:46.972884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 770/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:49.006672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71173e6e-5f3d-4479-b82f-de5a73089369 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:49.007251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71173e6e-5f3d-4479-b82f-de5a73089369 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:49.007251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71173e6e-5f3d-4479-b82f-de5a73089369 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:49.008655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:49.014978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ac589-3d19-40b8-9259-17c42d82ae1e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:49.015452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ac589-3d19-40b8-9259-17c42d82ae1e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:49.015560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ac589-3d19-40b8-9259-17c42d82ae1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:49.040323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-567ac589-3d19-40b8-9259-17c42d82ae1e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:49.041526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-567ac589-3d19-40b8-9259-17c42d82ae1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:49.051121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:49.091760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71173e6e-5f3d-4479-b82f-de5a73089369 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:49.103906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71173e6e-5f3d-4479-b82f-de5a73089369 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:49.107964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657776d-baba-43ba-871f-7ea2c66d4778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:49.108629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657776d-baba-43ba-871f-7ea2c66d4778 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:49.108909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657776d-baba-43ba-871f-7ea2c66d4778 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:49.109179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657776d-baba-43ba-871f-7ea2c66d4778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:49.118550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657776d-baba-43ba-871f-7ea2c66d4778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:49.120365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657776d-baba-43ba-871f-7ea2c66d4778 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:49.129824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 762/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:52.668492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00472c7-7779-4b2f-95da-94392aa39667 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:52.669382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00472c7-7779-4b2f-95da-94392aa39667 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:52.669382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00472c7-7779-4b2f-95da-94392aa39667 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:52.673927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:48:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:48:52.680403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d86d9-3d12-44f0-8fd4-18a6a3225991 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:52.681048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d86d9-3d12-44f0-8fd4-18a6a3225991 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:52.681294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d86d9-3d12-44f0-8fd4-18a6a3225991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:52.708526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-5e5d86d9-3d12-44f0-8fd4-18a6a3225991 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:52.709736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5e5d86d9-3d12-44f0-8fd4-18a6a3225991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:52.718914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:48:53.033567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00472c7-7779-4b2f-95da-94392aa39667 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:53.037324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d00472c7-7779-4b2f-95da-94392aa39667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:53.102098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00472c7-7779-4b2f-95da-94392aa39667 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:53.106667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f0d7-b879-45a5-a69f-81c5a90e01b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:53.107421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f0d7-b879-45a5-a69f-81c5a90e01b1 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:53.107703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f0d7-b879-45a5-a69f-81c5a90e01b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:53.107979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f0d7-b879-45a5-a69f-81c5a90e01b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:53.117709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f0d7-b879-45a5-a69f-81c5a90e01b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:53.119160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f0d7-b879-45a5-a69f-81c5a90e01b1 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:53.128987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 773/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:48:55.239275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76a30b-531a-4501-9e0f-2a618613ae6e None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:55.239881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76a30b-531a-4501-9e0f-2a618613ae6e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:55.239881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d76a30b-531a-4501-9e0f-2a618613ae6e None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:55.241233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 15:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:48:55.250430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a110823-b2ad-4102-93dc-7a60f1ab9e99 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:55.251183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a110823-b2ad-4102-93dc-7a60f1ab9e99 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:55.251366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a110823-b2ad-4102-93dc-7a60f1ab9e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:55.276899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6a110823-b2ad-4102-93dc-7a60f1ab9e99 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:48:55.278136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6a110823-b2ad-4102-93dc-7a60f1ab9e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:55.287570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:48:55.594182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76a30b-531a-4501-9e0f-2a618613ae6e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:48:55.596019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5d76a30b-531a-4501-9e0f-2a618613ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:48:55.604145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=65 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:48:55.610498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d76a30b-531a-4501-9e0f-2a618613ae6e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:48:55.614396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ea618-5334-4594-ab25-5fde4744d97a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:48:55.615031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea618-5334-4594-ab25-5fde4744d97a service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:48:55.615310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea618-5334-4594-ab25-5fde4744d97a service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:48:55.615587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ea618-5334-4594-ab25-5fde4744d97a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:48:55.625312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea618-5334-4594-ab25-5fde4744d97a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:48:55.626765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ea618-5334-4594-ab25-5fde4744d97a service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:48:55.636307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 765/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:48:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:49:05.270175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=32 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:49:05.632406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:24.290675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048cfed-4ab8-4744-a5e1-2d83ddd4a2d8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.291774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048cfed-4ab8-4744-a5e1-2d83ddd4a2d8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.292332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048cfed-4ab8-4744-a5e1-2d83ddd4a2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.298366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076d3c3-1d10-4327-9947-66298c1d9130 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.300726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076d3c3-1d10-4327-9947-66298c1d9130 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.300726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076d3c3-1d10-4327-9947-66298c1d9130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.318377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e048cfed-4ab8-4744-a5e1-2d83ddd4a2d8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.319593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e048cfed-4ab8-4744-a5e1-2d83ddd4a2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.327858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7076d3c3-1d10-4327-9947-66298c1d9130 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.329199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7076d3c3-1d10-4327-9947-66298c1d9130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.340482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 775/1540] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.348789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 766/1541] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:24.349615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc154b1-7a64-45de-83c0-d91fe4c3174f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.349908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc154b1-7a64-45de-83c0-d91fe4c3174f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.350197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc154b1-7a64-45de-83c0-d91fe4c3174f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.358637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ed80-d7da-4015-91cf-1ce8cbdabdf9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.358918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ed80-d7da-4015-91cf-1ce8cbdabdf9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.359200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9ed80-d7da-4015-91cf-1ce8cbdabdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.379800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7cc154b1-7a64-45de-83c0-d91fe4c3174f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.381057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7cc154b1-7a64-45de-83c0-d91fe4c3174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.387105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-43d9ed80-d7da-4015-91cf-1ce8cbdabdf9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.388318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-43d9ed80-d7da-4015-91cf-1ce8cbdabdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.397733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 776/1542] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:24.399371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63212fcb-ee4c-4336-afc3-aa3c86933ad9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.399814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63212fcb-ee4c-4336-afc3-aa3c86933ad9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.404348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63212fcb-ee4c-4336-afc3-aa3c86933ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.407661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 767/1543] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.408539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9ed80-d7da-4015-91cf-1ce8cbdabdf9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.420624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9ed80-d7da-4015-91cf-1ce8cbdabdf9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.436025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.436025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.436025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.436025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.436511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:24.436511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.436511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING py.warnings [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:50:24.436511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: warnings.warn(msg) Jun 30 15:50:24.436511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: Jun 30 15:50:24.436511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.439309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-63212fcb-ee4c-4336-afc3-aa3c86933ad9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.440801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-63212fcb-ee4c-4336-afc3-aa3c86933ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.449182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 768/1544] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:24.450532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.457406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 777/1545] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.458821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbbbaf-d8b6-4024-b153-e4346f472db9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.459324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbbbaf-d8b6-4024-b153-e4346f472db9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.459828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbbbaf-d8b6-4024-b153-e4346f472db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.463461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7d69e-acb7-4dbf-8aba-3fc7fcdbee84 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.469115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea2b5d-33f3-4772-b170-9e21d331d6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.469767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2b5d-33f3-4772-b170-9e21d331d6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.470225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2b5d-33f3-4772-b170-9e21d331d6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.470661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea2b5d-33f3-4772-b170-9e21d331d6f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.471786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea2b5d-33f3-4772-b170-9e21d331d6f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:24.472290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea2b5d-33f3-4772-b170-9e21d331d6f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.474078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea2b5d-33f3-4772-b170-9e21d331d6f2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.489157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 769/1546] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:24.490834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf55ff1-4869-4aef-bfa6-04634c4fbdbc None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.491452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf55ff1-4869-4aef-bfa6-04634c4fbdbc None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.491954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf55ff1-4869-4aef-bfa6-04634c4fbdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.493990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-fedbbbaf-d8b6-4024-b153-e4346f472db9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.495346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fedbbbaf-d8b6-4024-b153-e4346f472db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.512512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 778/1547] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.513471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbbbaf-d8b6-4024-b153-e4346f472db9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.515827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1bf55ff1-4869-4aef-bfa6-04634c4fbdbc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.517373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1bf55ff1-4869-4aef-bfa6-04634c4fbdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.526286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbbbaf-d8b6-4024-b153-e4346f472db9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.528897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 770/1548] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.529164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be8f00-45e8-4cbe-97e7-0ce2bd6920d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.530023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be8f00-45e8-4cbe-97e7-0ce2bd6920d7 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.530242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746384-8bfb-46d7-bdf5-ef3a03982214 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.530366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be8f00-45e8-4cbe-97e7-0ce2bd6920d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.530533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746384-8bfb-46d7-bdf5-ef3a03982214 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.530639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be8f00-45e8-4cbe-97e7-0ce2bd6920d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.530804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00746384-8bfb-46d7-bdf5-ef3a03982214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.531299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be8f00-45e8-4cbe-97e7-0ce2bd6920d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.532354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-a0be8f00-45e8-4cbe-97e7-0ce2bd6920d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:50:24.532354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:50:24.532354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:50:24.533595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be8f00-45e8-4cbe-97e7-0ce2bd6920d7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.552602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-00746384-8bfb-46d7-bdf5-ef3a03982214 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.554573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-00746384-8bfb-46d7-bdf5-ef3a03982214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.565126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 771/1549] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.565906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00746384-8bfb-46d7-bdf5-ef3a03982214 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.570957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 779/1550] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:24.572634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30185a46-a73f-4f57-896f-311d3daf377c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.572923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30185a46-a73f-4f57-896f-311d3daf377c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.573199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30185a46-a73f-4f57-896f-311d3daf377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.575958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00746384-8bfb-46d7-bdf5-ef3a03982214 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.591598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-30185a46-a73f-4f57-896f-311d3daf377c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.592854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-30185a46-a73f-4f57-896f-311d3daf377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.631908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.632595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.632876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.633153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.633840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.635774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.652312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 780/1551] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.653722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131d03-2f83-4978-be78-c485d0004434 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.654048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131d03-2f83-4978-be78-c485d0004434 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.654327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131d03-2f83-4978-be78-c485d0004434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.667168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 772/1552] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:24.667168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.672284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d6131d03-2f83-4978-be78-c485d0004434 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.673485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d6131d03-2f83-4978-be78-c485d0004434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.725110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc97e17-ad21-4946-9658-4e8bf0887029 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.729828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d36ec7d-4686-4301-80ea-d97f8b37de1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.730491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36ec7d-4686-4301-80ea-d97f8b37de1f admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.730774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36ec7d-4686-4301-80ea-d97f8b37de1f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.730934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 781/1553] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.732849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d36ec7d-4686-4301-80ea-d97f8b37de1f admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.732849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36ec7d-4686-4301-80ea-d97f8b37de1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:24.733063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee924c9-041c-4218-aed6-b698438d6c3e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.733149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36ec7d-4686-4301-80ea-d97f8b37de1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.733226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee924c9-041c-4218-aed6-b698438d6c3e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.733226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee924c9-041c-4218-aed6-b698438d6c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.734090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d36ec7d-4686-4301-80ea-d97f8b37de1f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.746634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 773/1554] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:24.747794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8882cd-252c-4adc-9c23-abc97b248245 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.748118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8882cd-252c-4adc-9c23-abc97b248245 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.748396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8882cd-252c-4adc-9c23-abc97b248245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.753392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-dee924c9-041c-4218-aed6-b698438d6c3e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.754624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-dee924c9-041c-4218-aed6-b698438d6c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.764022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 782/1555] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.765077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eaf0e-f3b6-4a96-90aa-c27edc877cac None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.765361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eaf0e-f3b6-4a96-90aa-c27edc877cac None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.765633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eaf0e-f3b6-4a96-90aa-c27edc877cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.767669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ac8882cd-252c-4adc-9c23-abc97b248245 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.768928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ac8882cd-252c-4adc-9c23-abc97b248245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.780556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 774/1556] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.781129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8882cd-252c-4adc-9c23-abc97b248245 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.790729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c78eaf0e-f3b6-4a96-90aa-c27edc877cac None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.791732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8882cd-252c-4adc-9c23-abc97b248245 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.792118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c78eaf0e-f3b6-4a96-90aa-c27edc877cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.796451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.797111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.797387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.797654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.798340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.799349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING py.warnings [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:50:24.799349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: warnings.warn(msg) Jun 30 15:50:24.799349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: Jun 30 15:50:24.800614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.801153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 783/1557] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.802399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac2c11-9c2c-4f2b-be43-539e5eb2d486 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.802685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac2c11-9c2c-4f2b-be43-539e5eb2d486 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.802962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac2c11-9c2c-4f2b-be43-539e5eb2d486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.821606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5aac2c11-9c2c-4f2b-be43-539e5eb2d486 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.823456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5aac2c11-9c2c-4f2b-be43-539e5eb2d486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.825233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 775/1558] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:24.826148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.831603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:24.836669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bf17-354a-49b9-9076-3c530fde0877 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.836799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 784/1559] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.837896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac2c11-9c2c-4f2b-be43-539e5eb2d486 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.842230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.842944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.843454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.843737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.844651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.847322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.848769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac2c11-9c2c-4f2b-be43-539e5eb2d486 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.853290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04f60d-03ae-4120-b5d5-e23ca48564d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.853940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04f60d-03ae-4120-b5d5-e23ca48564d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.854224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04f60d-03ae-4120-b5d5-e23ca48564d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.854498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04f60d-03ae-4120-b5d5-e23ca48564d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.855225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04f60d-03ae-4120-b5d5-e23ca48564d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:24.855528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04f60d-03ae-4120-b5d5-e23ca48564d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.857227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04f60d-03ae-4120-b5d5-e23ca48564d3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.872808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 776/1560] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:24.872808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.873065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 785/1561] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:24.873065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25509bae-9bf7-4698-88fd-18480c68246f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.873065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25509bae-9bf7-4698-88fd-18480c68246f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.873303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25509bae-9bf7-4698-88fd-18480c68246f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.885394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b21b6-219f-4b8c-98ba-fdba69e71fe6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.889860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.890733 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.891010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.891280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.891958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.892468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-25509bae-9bf7-4698-88fd-18480c68246f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.893655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-25509bae-9bf7-4698-88fd-18480c68246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.893865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:24.902414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 786/1562] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.903634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc844e-44c9-415b-90d6-aaf34f346dd4 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.903921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc844e-44c9-415b-90d6-aaf34f346dd4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.904232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbc844e-44c9-415b-90d6-aaf34f346dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.922794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-dbbc844e-44c9-415b-90d6-aaf34f346dd4 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:24.923980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-dbbc844e-44c9-415b-90d6-aaf34f346dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:24.924126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 777/1563] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:24.925054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.933464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:24.984455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 787/1564] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:24.985444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc844e-44c9-415b-90d6-aaf34f346dd4 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:24.989447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b22b1c-3eab-450d-8cee-0d0f993486e1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.994331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:24.995107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:24.995302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbc844e-44c9-415b-90d6-aaf34f346dd4 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:24.995424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:24.995641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:24.996406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:24.996750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:24.998310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.000113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.000790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.001071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.001346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.002073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.003690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.010042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 778/1565] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:25.011038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.018411 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 788/1566] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.018411 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.020820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62dd094-491b-4588-b561-903f93f33ebf admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.025694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.026338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.026622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.026895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.027850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7065b8d-8871-4e55-a17b-bb6fb3fc9901 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.027980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:25.028165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.029800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.032500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.033083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.033360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.033634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.034344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.035935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.042165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 779/1567] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:25.046304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 789/1568] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.047317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.048895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.057630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aac5d7-1f7d-4cb7-801d-b41b9d4f7564 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.058849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cab58ab-c95c-481d-8279-dd7fec804cf0 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.062395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.063069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.063319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.063610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.063739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.064287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.064417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.064569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.064844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.065553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:25.065859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.066454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.067474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.079760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 790/1569] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:25.080877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.092001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545bcb6-b530-461c-b984-5b9c72591049 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.093630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 780/1570] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.094384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.151765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.152473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.152757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.153036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.153735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.155694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.155819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b1a79-de07-4c5e-b6f4-f41debc7565a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.160466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.161155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.161440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.161714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.162433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:25.162740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.164369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.176604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 791/1571] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.184038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.184218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 781/1572] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:25.184218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.189332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58e8a2-1fdc-494b-915d-5a98c7f2ac77 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.189629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca1126-fad7-446a-9e19-5e174e784cc7 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.193853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.194496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.194612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.194785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.195058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.195238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.195520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.195772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.195890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.196531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.197393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.198582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.208527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 782/1573] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.212417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.218646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:25.223198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bdf151-3637-4dbc-a711-cb6093cfc4d1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.226586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 792/1574] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.227505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.229217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.229928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.230487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.231071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.232338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.234264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.247847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 783/1575] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.247847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.254664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687646d-a0e4-4c3c-8523-d52fadda76c3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.283253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:25.299961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3f260-249f-4c58-aefb-4bde712382f9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.304764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.305432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.305715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.305989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.306687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.308630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.312286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.313267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.313576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.313833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.314526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.316848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.334393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 793/1576] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.335278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.347286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 784/1577] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.347286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.401732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5a568-21bb-43c4-9333-0f34f1136a96 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.406465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.407079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88723608-bf26-4eec-ba93-22b05c2d9e09 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.407201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.407410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.407696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.408427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.410376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.411935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.412628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.412911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.413184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.413937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.415535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.429312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 785/1578] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.429312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.439063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 794/1579] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.439063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.439314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2861ab-9ebc-477d-a880-5e365c5e5eb7 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.442863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.443625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.443911 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.444214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.444905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.446679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224cad3-e49e-4d27-8e56-bcf2819b3d44 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.446812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.451376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.452306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.452586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.452858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] PATH_INFO: `/v3/projects/36b41d512e874482baa763f57a0359bd/users/3e7cd7f1f2fe4b63945d86361f722263/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.468726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36b41d512e874482baa763f57a0359bd, user_id=3e7cd7f1f2fe4b63945d86361f722263, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.469984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:50:25.469984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:50:25.469984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:50:25.472320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.472706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 786/1580] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.476153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.487611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 795/1581] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:25 2022] PUT /identity/v3/projects/36b41d512e874482baa763f57a0359bd/users/3e7cd7f1f2fe4b63945d86361f722263/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:25.491262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.526069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:25.537615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2edb8-00c5-48ad-881e-29ddca1450b2 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.550164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39f41-b9b0-40b3-8120-6ea12f66f120 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.554711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.555612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.555948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.556263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] PATH_INFO: `/v3/projects/4b65df0cd27d479e9bc39ecb95ff04d7/users/ead6569ad30b496092f49b3fe0c8a706/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.571134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b65df0cd27d479e9bc39ecb95ff04d7, user_id=ead6569ad30b496092f49b3fe0c8a706, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.573810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.588333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 796/1582] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:25 2022] PUT /identity/v3/projects/4b65df0cd27d479e9bc39ecb95ff04d7/users/ead6569ad30b496092f49b3fe0c8a706/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:25.589303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.595037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.595717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.596011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.596308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.597005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.598933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.610588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:25.623460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 787/1583] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/users => generated 284 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.623460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.631043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6390f-f524-4698-b203-8708a2099410 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.663022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b715453-b6e2-4a29-bc97-7b85cb931aca admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.668264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.668955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.669241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.669541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.670228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.672236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.693205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.693530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 797/1584] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.693879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.694164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.694492 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.695190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.697199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.697334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.707194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ce0fb-0304-4dde-a108-fc9040db186e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.711782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.712448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.712727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.713005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.713724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.715306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.719835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 788/1585] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.719835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.727167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1391-f808-4984-9730-48f6918fc03c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.730136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 798/1586] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.731020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.733189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c5400-3539-4607-8b6b-12d5f72b4de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.733908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5400-3539-4607-8b6b-12d5f72b4de5 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.734187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5400-3539-4607-8b6b-12d5f72b4de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.734455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c5400-3539-4607-8b6b-12d5f72b4de5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.735359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c5400-3539-4607-8b6b-12d5f72b4de5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.737387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c5400-3539-4607-8b6b-12d5f72b4de5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.741066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1fedf-f06c-4608-ba50-8320b20fcc1e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.746393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c042d5-a8e4-42fa-b805-3fc5d1a920b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.747051 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c042d5-a8e4-42fa-b805-3fc5d1a920b0 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.747329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c042d5-a8e4-42fa-b805-3fc5d1a920b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.747602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c042d5-a8e4-42fa-b805-3fc5d1a920b0 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.748326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c042d5-a8e4-42fa-b805-3fc5d1a920b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.748453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 789/1587] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.750499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c042d5-a8e4-42fa-b805-3fc5d1a920b0 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.752702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac7538-2d5f-4904-8b7f-38e084c7f0fa None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.752984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac7538-2d5f-4904-8b7f-38e084c7f0fa None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.753253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac7538-2d5f-4904-8b7f-38e084c7f0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.771207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e0ac7538-2d5f-4904-8b7f-38e084c7f0fa None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:25.771656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 799/1588] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/users => generated 283 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:25.772520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e0ac7538-2d5f-4904-8b7f-38e084c7f0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:25.772963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffaf48-590f-4367-af08-da0b736914ef None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.773249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffaf48-590f-4367-af08-da0b736914ef None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.773527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffaf48-590f-4367-af08-da0b736914ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.781058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 790/1589] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:25.781871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac7538-2d5f-4904-8b7f-38e084c7f0fa None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.791653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac7538-2d5f-4904-8b7f-38e084c7f0fa None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.792127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-84ffaf48-590f-4367-af08-da0b736914ef None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:25.793353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-84ffaf48-590f-4367-af08-da0b736914ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:25.796257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.796954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.797187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.797456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.798177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.799767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.801423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 800/1590] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:25.802203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffaf48-590f-4367-af08-da0b736914ef None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.811645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffaf48-590f-4367-af08-da0b736914ef None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.813532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 791/1591] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.816557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.817181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.817294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.817473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.817742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.818587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.820229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.827053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e398ebb-2399-442b-96ff-73a58ee38305 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.833368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 801/1592] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.833368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.833638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.833638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.833638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.833638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.834014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.834985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.843221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4ff81-f219-4c5d-be1d-87ce3a06214a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.845516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 792/1593] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.846333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.848098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.849302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.849580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.849845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] PATH_INFO: `/v3/projects/5418867eec2d44f6bfe12302db443446/users/d56405a1728a4229a128a0b09f4bce83/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.856239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f7c65-c06a-4f48-b517-957b78f051c2 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.861747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.862410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.862700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.862961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.863667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.864668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5418867eec2d44f6bfe12302db443446, user_id=d56405a1728a4229a128a0b09f4bce83, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.865332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.867608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.880472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 793/1594] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.880472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.881244 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 802/1595] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:25 2022] PUT /identity/v3/projects/5418867eec2d44f6bfe12302db443446/users/d56405a1728a4229a128a0b09f4bce83/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:25.882036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.888005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a8a5-ffcf-4ca0-990d-ccef5b4fec2c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.913213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:25.941546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115c38b-4a27-4b54-ba67-08d009b8f382 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.944166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.944858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.945078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.945204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.945403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.945722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.945996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.946153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.946256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.947763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.955415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.956917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.957707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 794/1596] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:25.958530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.960173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:25.967343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 803/1597] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:25.968688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:25.972021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f07815-8b08-4989-ae1b-d4255c54fb7e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.976824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.977593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.977883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.978159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.981932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faedf2-33b8-447d-8246-6b3577c0c930 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:25.987004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:25.987678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:25.987957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:25.988253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:25.988982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.990489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:25.990969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:25.992729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.005575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 804/1598] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:26.005575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.005821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 795/1599] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:26.005821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.016569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c77b09-e64a-40ac-ba27-55583a871086 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.020893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f664bf-f983-481d-8f4a-dca2ae99d7f9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.021335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.022115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.022460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.022892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.023824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.025600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80543e-f309-4d12-a17b-02d79c1ed7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.026323 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.026605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80543e-f309-4d12-a17b-02d79c1ed7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.026930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80543e-f309-4d12-a17b-02d79c1ed7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.028229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80543e-f309-4d12-a17b-02d79c1ed7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.029579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80543e-f309-4d12-a17b-02d79c1ed7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.031456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80543e-f309-4d12-a17b-02d79c1ed7f0 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.041021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 796/1600] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:26.041021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.050083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 805/1601] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:26.052246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c2e0d-a136-4487-ba21-520616c24fca None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.052538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c2e0d-a136-4487-ba21-520616c24fca None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.052823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c2e0d-a136-4487-ba21-520616c24fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.054825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd83c61-c329-4c3c-b718-c1663b1b9c10 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.061102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.062720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.063316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.063800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] PATH_INFO: `/v3/projects/f88ea20d31df42ec9e28d8f283aa15ec/users/7f2f23e763704ce2b0969e24ffde9243/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.082699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88ea20d31df42ec9e28d8f283aa15ec, user_id=7f2f23e763704ce2b0969e24ffde9243, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.086095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.086410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-246c2e0d-a136-4487-ba21-520616c24fca None None] MFA Rules not processed for user `d56405a1728a4229a128a0b09f4bce83`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:26.087736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-246c2e0d-a136-4487-ba21-520616c24fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:26.101276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:26.110385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 797/1602] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:26 2022] PUT /identity/v3/projects/f88ea20d31df42ec9e28d8f283aa15ec/users/7f2f23e763704ce2b0969e24ffde9243/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:26.114474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.175344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 806/1603] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:26.176445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c2e0d-a136-4487-ba21-520616c24fca None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.191197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764317b-b7c7-4d1e-923b-7620e5fde68b admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.196542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.197253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.197498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.197775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.198765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.201543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.214366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 798/1604] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:26.215043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c2e0d-a136-4487-ba21-520616c24fca None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.215179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.227150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0892a1c-045d-480a-afbb-fbbc1f715bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.229153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29970e52-b3b2-4cf5-b36d-f7959502b691 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.229289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0892a1c-045d-480a-afbb-fbbc1f715bda admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.230179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0892a1c-045d-480a-afbb-fbbc1f715bda admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.231381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0892a1c-045d-480a-afbb-fbbc1f715bda admin admin] PATH_INFO: `/v3/projects/ef2a47fe979b435a9352e23633aca709/users/7b02e79d268e4ccfa65d15eb6c929520/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.234768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.235837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.236154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.236426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] PATH_INFO: `/v3/projects/9874821533c3446c9d71a65fb5b8e703/users/2f3c67a263b14158a01fd906a31696cd/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.256983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9874821533c3446c9d71a65fb5b8e703, user_id=2f3c67a263b14158a01fd906a31696cd, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.260824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.262410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0892a1c-045d-480a-afbb-fbbc1f715bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2a47fe979b435a9352e23633aca709, user_id=7b02e79d268e4ccfa65d15eb6c929520, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.271222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0892a1c-045d-480a-afbb-fbbc1f715bda admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.285513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 799/1605] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:26 2022] PUT /identity/v3/projects/9874821533c3446c9d71a65fb5b8e703/users/2f3c67a263b14158a01fd906a31696cd/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:26.286110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.296906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 807/1606] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:26 2022] PUT /identity/v3/projects/ef2a47fe979b435a9352e23633aca709/users/7b02e79d268e4ccfa65d15eb6c929520/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:26.298609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f801f-f5c7-42c4-80f0-31ec17d30128 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.299176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f801f-f5c7-42c4-80f0-31ec17d30128 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.299398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f801f-f5c7-42c4-80f0-31ec17d30128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.329688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6e9f801f-f5c7-42c4-80f0-31ec17d30128 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:26.330693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6e9f801f-f5c7-42c4-80f0-31ec17d30128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:26.340081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:26.373160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a62b7-196e-48cc-9915-9127e2808fe6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.428426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 808/1607] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:26.430116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c0dd6-0b57-48c7-9dce-bdfdbbd51b23 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.430508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c0dd6-0b57-48c7-9dce-bdfdbbd51b23 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.430898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c0dd6-0b57-48c7-9dce-bdfdbbd51b23 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.437946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 809/1608] 10.222.0.50 () {54 vars in 961 bytes} [Thu Jun 30 15:50:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:50:26.439375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c0dd6-0b57-48c7-9dce-bdfdbbd51b23 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.457856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c0dd6-0b57-48c7-9dce-bdfdbbd51b23 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.467207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57970145-6b7f-4cb0-90bc-cf53a47137f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.468990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970145-6b7f-4cb0-90bc-cf53a47137f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.469733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970145-6b7f-4cb0-90bc-cf53a47137f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.470513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57970145-6b7f-4cb0-90bc-cf53a47137f6 admin admin] PATH_INFO: `/v3/projects/e59bf7d520e340948ec3cc41bb0598a2/users/14edcf9cb9834f4f87049adb7c193c31/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.477937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdfc9e1-f939-4280-8dff-4a48afbbf249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.479178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc9e1-f939-4280-8dff-4a48afbbf249 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.479519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc9e1-f939-4280-8dff-4a48afbbf249 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.479819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdfc9e1-f939-4280-8dff-4a48afbbf249 admin admin] PATH_INFO: `/v3/projects/01a8c235ed97450bb8518af9d7dc1983/users/d30ffdb0374c4600a09476cfb3a07cb4/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.495117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57970145-6b7f-4cb0-90bc-cf53a47137f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e59bf7d520e340948ec3cc41bb0598a2, user_id=14edcf9cb9834f4f87049adb7c193c31, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.498177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57970145-6b7f-4cb0-90bc-cf53a47137f6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.500671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfc9e1-f939-4280-8dff-4a48afbbf249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a8c235ed97450bb8518af9d7dc1983, user_id=d30ffdb0374c4600a09476cfb3a07cb4, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.503625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdfc9e1-f939-4280-8dff-4a48afbbf249 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.520655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 810/1609] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:26 2022] PUT /identity/v3/projects/e59bf7d520e340948ec3cc41bb0598a2/users/14edcf9cb9834f4f87049adb7c193c31/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:26.521834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43f679-97ac-4f51-a1f3-176a674aa24e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.522339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43f679-97ac-4f51-a1f3-176a674aa24e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.522712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43f679-97ac-4f51-a1f3-176a674aa24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.535946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 800/1610] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:26 2022] PUT /identity/v3/projects/01a8c235ed97450bb8518af9d7dc1983/users/d30ffdb0374c4600a09476cfb3a07cb4/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:26.535946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4a82e-e958-4a7c-b987-45fba9924238 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.535946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4a82e-e958-4a7c-b987-45fba9924238 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.535946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4a82e-e958-4a7c-b987-45fba9924238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.541889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1c43f679-97ac-4f51-a1f3-176a674aa24e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:26.543402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1c43f679-97ac-4f51-a1f3-176a674aa24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:26.569772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-cbb4a82e-e958-4a7c-b987-45fba9924238 None None] MFA Rules not processed for user `7b02e79d268e4ccfa65d15eb6c929520`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:26.571265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-cbb4a82e-e958-4a7c-b987-45fba9924238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:26.616558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 811/1611] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:26.618575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43f679-97ac-4f51-a1f3-176a674aa24e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.622523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:26.655693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 801/1612] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:26.657219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4505556-7eb6-4b5a-9700-856a3ba8202d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.657557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4505556-7eb6-4b5a-9700-856a3ba8202d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.658053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4505556-7eb6-4b5a-9700-856a3ba8202d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.691582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f4505556-7eb6-4b5a-9700-856a3ba8202d None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:26.693092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f4505556-7eb6-4b5a-9700-856a3ba8202d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:26.698302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43f679-97ac-4f51-a1f3-176a674aa24e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.703805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda6a8a1-77df-44b5-ba5c-23a660913bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.704743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6a8a1-77df-44b5-ba5c-23a660913bfc service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.705658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6a8a1-77df-44b5-ba5c-23a660913bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.706037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda6a8a1-77df-44b5-ba5c-23a660913bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.720388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6a8a1-77df-44b5-ba5c-23a660913bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.722151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda6a8a1-77df-44b5-ba5c-23a660913bfc service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.736990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 812/1613] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:26.738643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2615d10-582e-4d89-a406-eaa73982d72d None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.738937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2615d10-582e-4d89-a406-eaa73982d72d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.739371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2615d10-582e-4d89-a406-eaa73982d72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.785420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f2615d10-582e-4d89-a406-eaa73982d72d None None] MFA Rules not processed for user `14edcf9cb9834f4f87049adb7c193c31`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:26.787847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f2615d10-582e-4d89-a406-eaa73982d72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:26.802176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:26.826337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 802/1614] 10.222.0.50 () {58 vars in 1058 bytes} [Thu Jun 30 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:26.836541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b93e-5f86-4235-97f8-b6ba987bde3c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.836833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b93e-5f86-4235-97f8-b6ba987bde3c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.837782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6b93e-5f86-4235-97f8-b6ba987bde3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.865841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-37b6b93e-5f86-4235-97f8-b6ba987bde3c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:26.867669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-37b6b93e-5f86-4235-97f8-b6ba987bde3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:26.881978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 803/1615] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:26.883402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b93e-5f86-4235-97f8-b6ba987bde3c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.900280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6b93e-5f86-4235-97f8-b6ba987bde3c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.908429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.909173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.909460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.909732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:26.934331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:26.936511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:26.956546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 813/1616] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:26.956776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 804/1617] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:26.957385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.958176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2615d10-582e-4d89-a406-eaa73982d72d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:26.973174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936acc1-dba1-45a0-9950-6f9f25183bf5 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:26.980435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:26.982086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:26.982424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:26.982969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.057439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2615d10-582e-4d89-a406-eaa73982d72d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.069344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632750b7-9565-4c14-8ac0-cecec46bf124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.070315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632750b7-9565-4c14-8ac0-cecec46bf124 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.071410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632750b7-9565-4c14-8ac0-cecec46bf124 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.071685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632750b7-9565-4c14-8ac0-cecec46bf124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.080108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.083759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.086043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632750b7-9565-4c14-8ac0-cecec46bf124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.087754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632750b7-9565-4c14-8ac0-cecec46bf124 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.099968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 805/1618] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:27.101725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.105486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 814/1619] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:27.107764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250866c-4191-40ed-b221-fb1ac748257b None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.108077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250866c-4191-40ed-b221-fb1ac748257b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.108469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250866c-4191-40ed-b221-fb1ac748257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.133961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850f4ab-9740-427a-a24d-e4ff0a9e9846 service nova] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.144917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.146342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.148043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.149055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.173480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.190799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.194140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9250866c-4191-40ed-b221-fb1ac748257b None None] MFA Rules not processed for user `3e7cd7f1f2fe4b63945d86361f722263`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:27.199585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9250866c-4191-40ed-b221-fb1ac748257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:27.219236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 806/1620] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:27.220368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.240224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515c895-03e6-41f3-88bd-1f0a68a79726 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.250565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.251594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.251689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.251962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.252692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.254613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.286084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:27.315644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 807/1621] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:27 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:27.316283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.399029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 815/1622] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:27.400423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250866c-4191-40ed-b221-fb1ac748257b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.429655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250866c-4191-40ed-b221-fb1ac748257b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.520142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ebf5b-c80d-44ee-9b4b-0d1f7f225896 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.536200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e957e5a-de15-4be3-82be-cb4270a79469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.541305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957e5a-de15-4be3-82be-cb4270a79469 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.541837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957e5a-de15-4be3-82be-cb4270a79469 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.542752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e957e5a-de15-4be3-82be-cb4270a79469 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.543687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e957e5a-de15-4be3-82be-cb4270a79469 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.550313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e957e5a-de15-4be3-82be-cb4270a79469 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.588384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06d6e8-6d94-45ce-9fc7-c27f456c3d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.589809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06d6e8-6d94-45ce-9fc7-c27f456c3d3c admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.589809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06d6e8-6d94-45ce-9fc7-c27f456c3d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.590075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06d6e8-6d94-45ce-9fc7-c27f456c3d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.590848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06d6e8-6d94-45ce-9fc7-c27f456c3d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.604255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06d6e8-6d94-45ce-9fc7-c27f456c3d3c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.632675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 808/1623] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:27 2022] POST /identity/v3/users => generated 289 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:27.637933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9f824-1333-4c8f-87c0-6e26155ba366 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.642174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9f824-1333-4c8f-87c0-6e26155ba366 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.642581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9f824-1333-4c8f-87c0-6e26155ba366 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.648847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 809/1624] 10.222.0.50 () {54 vars in 912 bytes} [Thu Jun 30 15:50:27 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:50:27.649844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9f824-1333-4c8f-87c0-6e26155ba366 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.659929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 816/1625] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:27 2022] POST /identity/v3/projects => generated 405 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:27.663834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490dfbef-cf84-40ac-b95d-08b2c52dfa39 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.666570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490dfbef-cf84-40ac-b95d-08b2c52dfa39 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.667040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490dfbef-cf84-40ac-b95d-08b2c52dfa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.685485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9f824-1333-4c8f-87c0-6e26155ba366 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.709469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-490dfbef-cf84-40ac-b95d-08b2c52dfa39 None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:27.709976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-490dfbef-cf84-40ac-b95d-08b2c52dfa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:27.771745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:27.839756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 817/1626] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:50:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:27.840783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490dfbef-cf84-40ac-b95d-08b2c52dfa39 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.877963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.879999 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.880296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.880579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.884488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.889975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.897502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:27.915162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 810/1627] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:27 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:27.918161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.920342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490dfbef-cf84-40ac-b95d-08b2c52dfa39 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.925932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.926777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.927159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.927526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:27.928390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:27.930432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:27.960933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 818/1628] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:27 2022] POST /identity/v3/users => generated 309 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:27.960933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:27.974105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572520cd-ac30-4ce7-9624-6d9a221ede8f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:27.980479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:27.981634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:27.982020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:27.982436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] PATH_INFO: `/v3/projects/900afaf15f8d4adea6f0a8bf256105a5/users/4a8d4f65d2b24e4c96d8a9d1d21300d1/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.005916 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900afaf15f8d4adea6f0a8bf256105a5, user_id=4a8d4f65d2b24e4c96d8a9d1d21300d1, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.010096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.034669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 819/1629] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:27 2022] PUT /identity/v3/projects/900afaf15f8d4adea6f0a8bf256105a5/users/4a8d4f65d2b24e4c96d8a9d1d21300d1/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:28.036080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:28.052539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a947-b1ed-467f-bac7-44575a1a324e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.121499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.124536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.124981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.125267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.205864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f261868-2dcc-477f-84f1-dc5ae12a33e6 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.224269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.225972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.226205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.226482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.230089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.234397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.258738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 820/1630] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:28.263355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:28.284643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc05738-255f-464e-89c7-d1bb615e61cf admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.298470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.299167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.300596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.300860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.303127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.307883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.326707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.330971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.335649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 821/1631] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:28.336784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:28.362254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55f693-1a34-4737-acc8-b42d002c0e18 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.370572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 811/1632] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:28.375052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:28.390364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.395824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.399173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.399750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] PATH_INFO: `/v3/projects/cbd0ccc961d3403d98288f0f62c9f36d/users/85b248c8f292492fb9e4badfa6b483f9/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.407159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69710b4f-3f8c-4364-bce5-b07999ebde1a service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.417727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d6981-0b81-4700-8f1d-c0d3b0ae9723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.418645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6981-0b81-4700-8f1d-c0d3b0ae9723 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.418927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6981-0b81-4700-8f1d-c0d3b0ae9723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.419204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6981-0b81-4700-8f1d-c0d3b0ae9723 admin admin] PATH_INFO: `/v3/projects/900afaf15f8d4adea6f0a8bf256105a5/users/4a8d4f65d2b24e4c96d8a9d1d21300d1/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.425005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d6981-0b81-4700-8f1d-c0d3b0ae9723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900afaf15f8d4adea6f0a8bf256105a5, user_id=4a8d4f65d2b24e4c96d8a9d1d21300d1, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.432902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d6981-0b81-4700-8f1d-c0d3b0ae9723 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.452152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd0ccc961d3403d98288f0f62c9f36d, user_id=85b248c8f292492fb9e4badfa6b483f9, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.456966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 812/1633] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:28 2022] PUT /identity/v3/projects/900afaf15f8d4adea6f0a8bf256105a5/users/4a8d4f65d2b24e4c96d8a9d1d21300d1/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:28.463326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.470474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57651427-6cbd-4867-8745-da073aab27e5 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.470802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57651427-6cbd-4867-8745-da073aab27e5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.471608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57651427-6cbd-4867-8745-da073aab27e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.503961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 822/1634] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:28 2022] PUT /identity/v3/projects/cbd0ccc961d3403d98288f0f62c9f36d/users/85b248c8f292492fb9e4badfa6b483f9/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:28.512174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:28.530754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:28.536333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-57651427-6cbd-4867-8745-da073aab27e5 None None] MFA Rules not processed for user `4a8d4f65d2b24e4c96d8a9d1d21300d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:28.537690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-57651427-6cbd-4867-8745-da073aab27e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:28.661239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac0afe-2562-4f0e-b23e-77041ddada22 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.672494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.673187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.673468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.673741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.676853 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.679478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.704690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 823/1635] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:28.711650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:28.714788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 813/1636] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:28.735122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57651427-6cbd-4867-8745-da073aab27e5 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:28.735517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d644741-16d8-4e5d-adae-9a2042484600 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.777822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.779221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.780377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.781389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] PATH_INFO: `/v3/projects/cbd0ccc961d3403d98288f0f62c9f36d/users/85b248c8f292492fb9e4badfa6b483f9/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:28.787199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd0ccc961d3403d98288f0f62c9f36d, user_id=85b248c8f292492fb9e4badfa6b483f9, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:28.791362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:28.798451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:28.822026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 824/1637] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:28 2022] PUT /identity/v3/projects/cbd0ccc961d3403d98288f0f62c9f36d/users/85b248c8f292492fb9e4badfa6b483f9/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:28.880494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57651427-6cbd-4867-8745-da073aab27e5 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:28.959567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f96497-f829-4d13-9380-4a07cb752947 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:28.960553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f96497-f829-4d13-9380-4a07cb752947 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:28.961047 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f96497-f829-4d13-9380-4a07cb752947 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:28.961686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f96497-f829-4d13-9380-4a07cb752947 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:29.114661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f96497-f829-4d13-9380-4a07cb752947 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:29.131304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f96497-f829-4d13-9380-4a07cb752947 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:29.160298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 814/1638] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:29.424267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:29.440487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024eb00-c0ef-4488-8b68-f3e2565d29c9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:29.452733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaae4c6-7315-4539-8323-f1b42cd14a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:29.453404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae4c6-7315-4539-8323-f1b42cd14a0f service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:29.453685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae4c6-7315-4539-8323-f1b42cd14a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:29.453957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae4c6-7315-4539-8323-f1b42cd14a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:29.561578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae4c6-7315-4539-8323-f1b42cd14a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:29.563785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae4c6-7315-4539-8323-f1b42cd14a0f service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:29.576936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 825/1639] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:29.840566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9787b-f9d5-4c6c-b3f4-79d46cf6f911 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:29.841152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9787b-f9d5-4c6c-b3f4-79d46cf6f911 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:29.841152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9787b-f9d5-4c6c-b3f4-79d46cf6f911 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:29.843351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 815/1640] 10.222.0.50 () {54 vars in 961 bytes} [Thu Jun 30 15:50:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:50:29.867456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac20d-6668-453e-abb6-9762409d542c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:29.867784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac20d-6668-453e-abb6-9762409d542c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:29.868069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac20d-6668-453e-abb6-9762409d542c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:29.923026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-72eac20d-6668-453e-abb6-9762409d542c None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:29.927214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-72eac20d-6668-453e-abb6-9762409d542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:30.114589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 826/1641] 10.222.0.50 () {58 vars in 1058 bytes} [Thu Jun 30 15:50:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:30.122088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d722ee-05f5-4dc7-b502-83233a60fa0b None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:30.122507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d722ee-05f5-4dc7-b502-83233a60fa0b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:30.122793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d722ee-05f5-4dc7-b502-83233a60fa0b None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:30.125977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 816/1642] 10.222.0.50 () {54 vars in 912 bytes} [Thu Jun 30 15:50:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:50:30.134266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462cf50-cca0-42b8-98c4-9e5edf20b8cf None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:30.134852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462cf50-cca0-42b8-98c4-9e5edf20b8cf None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:30.135078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462cf50-cca0-42b8-98c4-9e5edf20b8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:30.164800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f462cf50-cca0-42b8-98c4-9e5edf20b8cf None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:30.166677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f462cf50-cca0-42b8-98c4-9e5edf20b8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:30.182576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 827/1643] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:50:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:30.198389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d722ee-05f5-4dc7-b502-83233a60fa0b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:30.208189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:30.219445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d722ee-05f5-4dc7-b502-83233a60fa0b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:30.223466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:30.225050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:30.225516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:30.225928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:30.239249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:30.241006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:30.269433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 817/1644] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:31.626758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de2e72-e1b7-4e77-8019-1c0a93b622af None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:31.627377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de2e72-e1b7-4e77-8019-1c0a93b622af None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:31.627377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de2e72-e1b7-4e77-8019-1c0a93b622af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:31.678048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-28de2e72-e1b7-4e77-8019-1c0a93b622af None None] MFA Rules not processed for user `85b248c8f292492fb9e4badfa6b483f9`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:31.679402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-28de2e72-e1b7-4e77-8019-1c0a93b622af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:31.858652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 828/1645] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:31 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:32.418181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:32.512727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c12c97-93c8-4ad5-9adc-570cb9be65c2 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:32.522008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:32.525630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:32.527091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:32.528045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] PATH_INFO: `/v3/users/d56405a1728a4229a128a0b09f4bce83` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:32.532036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56405a1728a4229a128a0b09f4bce83)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:32.540546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:32.703285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56405a1728a4229a128a0b09f4bce83'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:32.733157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56405a1728a4229a128a0b09f4bce83'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:32.750019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56405a1728a4229a128a0b09f4bce83'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:32.751872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56405a1728a4229a128a0b09f4bce83'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:32.752959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56405a1728a4229a128a0b09f4bce83'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:32.796588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7026-b08f-4e5d-88f4-004188022775 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:32.797102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7026-b08f-4e5d-88f4-004188022775 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:32.797175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b7026-b08f-4e5d-88f4-004188022775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:32.802850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 818/1646] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:32 2022] DELETE /identity/v3/users/d56405a1728a4229a128a0b09f4bce83 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:32.803294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:32.850381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-652b7026-b08f-4e5d-88f4-004188022775 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:32.853169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-652b7026-b08f-4e5d-88f4-004188022775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:33.044883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 829/1647] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:33.067171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:33.072526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:33.406730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf575924-0c69-4e4e-a6de-2aaf631d8088 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:33.415510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153a2b6-48b6-48e4-a7b8-1a6727a45206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:33.415865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153a2b6-48b6-48e4-a7b8-1a6727a45206 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:33.418739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153a2b6-48b6-48e4-a7b8-1a6727a45206 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:33.419412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153a2b6-48b6-48e4-a7b8-1a6727a45206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:33.494203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153a2b6-48b6-48e4-a7b8-1a6727a45206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:33.495703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153a2b6-48b6-48e4-a7b8-1a6727a45206 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:33.531083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 819/1648] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:34.328260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66285-d885-478d-88b7-8976ca6827cd None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:34.329574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66285-d885-478d-88b7-8976ca6827cd None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:34.330084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66285-d885-478d-88b7-8976ca6827cd None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:34.332292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 830/1649] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:50:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:50:34.342154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c4462-bde7-4818-b5bf-7f9c48ce3a97 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:34.342479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c4462-bde7-4818-b5bf-7f9c48ce3a97 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:34.342764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c4462-bde7-4818-b5bf-7f9c48ce3a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:34.383343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fc4c4462-bde7-4818-b5bf-7f9c48ce3a97 None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:34.392936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fc4c4462-bde7-4818-b5bf-7f9c48ce3a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:34.431341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:34.575254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 820/1650] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:34.959821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf82d-2f00-4ad2-883e-54d5ce568613 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:34.960043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf82d-2f00-4ad2-883e-54d5ce568613 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:34.960565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cf82d-2f00-4ad2-883e-54d5ce568613 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:34.962264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 831/1651] 10.222.0.50 () {54 vars in 960 bytes} [Thu Jun 30 15:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:50:34.970047 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:34.970359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:34.970612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:35.026459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:35.026459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:35.073726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf82d-2f00-4ad2-883e-54d5ce568613 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:35.074001 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 821/1652] 10.222.0.50 () {58 vars in 1057 bytes} [Thu Jun 30 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:35.099413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cf82d-2f00-4ad2-883e-54d5ce568613 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:35.138292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:35.138940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:35.139288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:35.139556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:35.154764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:35.158423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:35.185967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 832/1653] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:36.505563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:36.507035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:36.534572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af907e3-6ce7-4117-8c31-99a3b837ac29 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:36.539712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:36.540947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:36.541391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:36.541812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] PATH_INFO: `/v3/users/3e7cd7f1f2fe4b63945d86361f722263` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:36.543222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7cd7f1f2fe4b63945d86361f722263)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:36.547657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:36.642986 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7cd7f1f2fe4b63945d86361f722263'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:36.644636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:36.647326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:36.681582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7cd7f1f2fe4b63945d86361f722263'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:36.683853 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5e2fb-6354-47f3-9f18-08eec74313fd service placement] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:36.702428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7cd7f1f2fe4b63945d86361f722263'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:36.704659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:36.705692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:36.705979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:36.706254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] PATH_INFO: `/v3/projects/5418867eec2d44f6bfe12302db443446` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:36.706814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7cd7f1f2fe4b63945d86361f722263'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:36.709678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-53b0fc74-3370-445f-b79f-ff46ca879e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7cd7f1f2fe4b63945d86361f722263'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:36.714438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5418867eec2d44f6bfe12302db443446)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:36.733099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:36.801235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 822/1654] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:36 2022] DELETE /identity/v3/users/3e7cd7f1f2fe4b63945d86361f722263 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:36.988168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5418867eec2d44f6bfe12302db443446', 'request_id': 'req-0aa239ce-d374-48ed-a439-38218dcbed06'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:37.007228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5418867eec2d44f6bfe12302db443446', 'request_id': 'req-0aa239ce-d374-48ed-a439-38218dcbed06'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:37.021061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5418867eec2d44f6bfe12302db443446', 'request_id': 'req-0aa239ce-d374-48ed-a439-38218dcbed06'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:37.037437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] The token cache is being invalidate because project 5418867eec2d44f6bfe12302db443446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:37.038052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:37.040498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0aa239ce-d374-48ed-a439-38218dcbed06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:37.042961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 833/1655] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:36 2022] DELETE /identity/v3/projects/5418867eec2d44f6bfe12302db443446 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:50:37.047874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e6f49-4aba-43ce-8e24-a5056def3620 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:37.048464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e6f49-4aba-43ce-8e24-a5056def3620 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:37.048888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e6f49-4aba-43ce-8e24-a5056def3620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:37.084746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7e8e6f49-4aba-43ce-8e24-a5056def3620 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:37.086381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7e8e6f49-4aba-43ce-8e24-a5056def3620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:37.230249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 823/1656] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:37.231253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f9761-39b2-426b-82ba-e736393628a1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:37.231562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f9761-39b2-426b-82ba-e736393628a1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:37.231844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f9761-39b2-426b-82ba-e736393628a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:37.266795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-249f9761-39b2-426b-82ba-e736393628a1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:37.268073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-249f9761-39b2-426b-82ba-e736393628a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:37.293979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 834/1657] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:37.298804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e6f49-4aba-43ce-8e24-a5056def3620 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:37.315469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e6f49-4aba-43ce-8e24-a5056def3620 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:37.332644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:37.333351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:37.333635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:37.333910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:37.334659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:37.334968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:37.336667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:37.387347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 824/1658] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:37.388867 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f9761-39b2-426b-82ba-e736393628a1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:37.413067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f9761-39b2-426b-82ba-e736393628a1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:37.430641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:37.431382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:37.431644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:37.431925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:37.433774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:37.434946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:37.557096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 835/1659] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:37 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:37.557508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:37.568806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:37.601569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:37.611159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:37.613952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f779102-4ef3-45c9-a997-360db1063ebf admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:37.746133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:37.765873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:37.767692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:37.768662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:37.770314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:37.771371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:37.775885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:37.792992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e86c15-b96d-42f5-996b-12a0b6699eda admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:37.808155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:37.809952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:37.810539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:37.811340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] PATH_INFO: `/v3/projects/36b41d512e874482baa763f57a0359bd` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:37.820983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] RBAC: Authorizing `identity:delete_project(project_id=36b41d512e874482baa763f57a0359bd)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:37.840772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:37.843737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 825/1660] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:37 2022] POST /identity/v3/users => generated 287 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:37.847167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:37.869187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2c53e-7dc7-4376-a71a-9f7d92e91375 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:37.878478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:37.879592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:37.880098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:37.880524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:37.882379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:37.884250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:37.900216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 826/1661] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:37.910231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:37.924275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5752380-d4ad-42a2-b595-e3e10674c961 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:37.952869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36b41d512e874482baa763f57a0359bd', 'request_id': 'req-9c177853-85c2-47c7-90b8-3e9af477855e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:37.971524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36b41d512e874482baa763f57a0359bd', 'request_id': 'req-9c177853-85c2-47c7-90b8-3e9af477855e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.006952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36b41d512e874482baa763f57a0359bd', 'request_id': 'req-9c177853-85c2-47c7-90b8-3e9af477855e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.017529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:38.018279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:38.018563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:38.018838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:38.019559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:38.021267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:38.034384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 827/1662] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:38.043032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:38.047877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] The token cache is being invalidate because project 36b41d512e874482baa763f57a0359bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:38.049457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.059537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dca8e1-3f45-48ed-94e9-413c6a2d910e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:38.066050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceeae03-22b5-4bac-a537-baf64f4b9d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:38.067127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceeae03-22b5-4bac-a537-baf64f4b9d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:38.067522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceeae03-22b5-4bac-a537-baf64f4b9d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:38.067886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceeae03-22b5-4bac-a537-baf64f4b9d23 admin admin] PATH_INFO: `/v3/projects/121b500ff0bf4590ad411bc6101a9c12/users/4e6fc6c2524c4d5d876960e3da581baa/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:38.070713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.082381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 836/1663] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:37 2022] DELETE /identity/v3/projects/36b41d512e874482baa763f57a0359bd => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:38.086341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:38.088221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceeae03-22b5-4bac-a537-baf64f4b9d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121b500ff0bf4590ad411bc6101a9c12, user_id=4e6fc6c2524c4d5d876960e3da581baa, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:38.091933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceeae03-22b5-4bac-a537-baf64f4b9d23 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:38.108309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:38.115886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 828/1664] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:38 2022] PUT /identity/v3/projects/121b500ff0bf4590ad411bc6101a9c12/users/4e6fc6c2524c4d5d876960e3da581baa/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:38.129302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:38.130180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:38.130471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:38.170231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c177853-85c2-47c7-90b8-3e9af477855e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:38.175349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] MFA Rules not processed for user `4e6fc6c2524c4d5d876960e3da581baa`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:38.176895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:38.287947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:38.376996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 829/1665] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:38.384686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:38.386744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:38.391552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:38.391908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:38.392796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:38.393254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] PATH_INFO: `/v3/users/85b248c8f292492fb9e4badfa6b483f9` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:38.398938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b248c8f292492fb9e4badfa6b483f9)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:38.403514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: WARNING py.warnings [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:50:38.403514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: warnings.warn(msg) Jun 30 15:50:38.403514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: Jun 30 15:50:38.408008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:38.449983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e388b7-350b-41f7-b7a6-0167336043d3 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:38.479892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:38.481041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:38.481512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:38.481935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:38.483224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:38.496389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:38.619689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b248c8f292492fb9e4badfa6b483f9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.621458 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b248c8f292492fb9e4badfa6b483f9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.676389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 830/1666] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:38 2022] POST /identity/v3/projects => generated 369 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:38.679785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:38.684334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:38.695068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b248c8f292492fb9e4badfa6b483f9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.731810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b248c8f292492fb9e4badfa6b483f9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.737495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b248c8f292492fb9e4badfa6b483f9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:38.766191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:38.807759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 837/1667] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:38 2022] DELETE /identity/v3/users/85b248c8f292492fb9e4badfa6b483f9 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:38.961582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ac229-c550-4c46-83ff-49b25b92ca84 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:39.130353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:39.131058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.131330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.131591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.132346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:39.134327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:39.215430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 831/1668] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:38 2022] POST /identity/v3/users => generated 291 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:39.222211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:39.236127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbf038-2675-411c-b780-26bbaf32caef admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:39.248530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:39.249236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.249515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.249790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.250557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:39.252286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:39.289597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 838/1669] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:39.293765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:39.306084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d7e38-473c-4d3f-80d9-fd0e4c62160c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:39.320714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:39.321661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.321941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.322211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] PATH_INFO: `/v3/projects/56583df03be449cab79fda351ebeb904/users/2998758567cf48989dadb1cf2cbe3a48/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.356895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56583df03be449cab79fda351ebeb904, user_id=2998758567cf48989dadb1cf2cbe3a48, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:39.359671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:39.414011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 832/1670] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:39 2022] PUT /identity/v3/projects/56583df03be449cab79fda351ebeb904/users/2998758567cf48989dadb1cf2cbe3a48/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:39.417981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:39.434792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d1694-9730-4435-a249-c9e8646fa4f2 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:39.592198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee279715-12a1-4545-91c2-554095f05547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:39.593627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279715-12a1-4545-91c2-554095f05547 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.593915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279715-12a1-4545-91c2-554095f05547 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.594177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee279715-12a1-4545-91c2-554095f05547 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.597536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279715-12a1-4545-91c2-554095f05547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:39.599214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee279715-12a1-4545-91c2-554095f05547 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:39.628268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 839/1671] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:39.632423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:39.645679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.645974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.646253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.653731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3dbaf-4b3f-498f-9bf2-86cdd0453a19 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:39.664364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca36d8c-02ac-482b-923b-68dba4a8f0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:39.665447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36d8c-02ac-482b-923b-68dba4a8f0d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.665951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36d8c-02ac-482b-923b-68dba4a8f0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.666342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca36d8c-02ac-482b-923b-68dba4a8f0d6 admin admin] PATH_INFO: `/v3/projects/56583df03be449cab79fda351ebeb904/users/2998758567cf48989dadb1cf2cbe3a48/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.668923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36d8c-02ac-482b-923b-68dba4a8f0d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56583df03be449cab79fda351ebeb904, user_id=2998758567cf48989dadb1cf2cbe3a48, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:39.672446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca36d8c-02ac-482b-923b-68dba4a8f0d6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:39.688041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 833/1672] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:39 2022] PUT /identity/v3/projects/56583df03be449cab79fda351ebeb904/users/2998758567cf48989dadb1cf2cbe3a48/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:39.702064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589953-3827-40cb-87e6-5b5472640173 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.702685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589953-3827-40cb-87e6-5b5472640173 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.703671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38589953-3827-40cb-87e6-5b5472640173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.713351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] MFA Rules not processed for user `2f3c67a263b14158a01fd906a31696cd`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:39.714569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:39.768404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-38589953-3827-40cb-87e6-5b5472640173 None None] MFA Rules not processed for user `2998758567cf48989dadb1cf2cbe3a48`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:39.769950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-38589953-3827-40cb-87e6-5b5472640173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:39.785294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:39.828444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 840/1673] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:39.839493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:39.844070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:39.944673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 834/1674] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:39.956639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38589953-3827-40cb-87e6-5b5472640173 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:39.958367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-38589953-3827-40cb-87e6-5b5472640173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:39.959607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498bd98-3fb5-4578-9ab7-24d734554c28 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:39.982469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:39.986918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:39.987592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:39.987969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:39.995062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:39.999243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:40.001054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:40.013217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 841/1675] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:40.097835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38589953-3827-40cb-87e6-5b5472640173 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:40.109009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ed9ba-5aff-4552-99c8-07737f184cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:40.121255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ed9ba-5aff-4552-99c8-07737f184cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:40.121787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ed9ba-5aff-4552-99c8-07737f184cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:40.122190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ed9ba-5aff-4552-99c8-07737f184cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:40.154116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ed9ba-5aff-4552-99c8-07737f184cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:40.157345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ed9ba-5aff-4552-99c8-07737f184cb6 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:40.173996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 835/1676] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:39 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:40.680238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:40.681721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:40.832428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae06c2-6bb0-421c-9bb8-45286c3295ad service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:40.845446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:40.846549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:40.846809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:40.847153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] PATH_INFO: `/v3/users/7b02e79d268e4ccfa65d15eb6c929520` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:40.848578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b02e79d268e4ccfa65d15eb6c929520)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:40.850710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:40.949674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b02e79d268e4ccfa65d15eb6c929520'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:40.953506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b02e79d268e4ccfa65d15eb6c929520'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:40.984889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b02e79d268e4ccfa65d15eb6c929520'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.010988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b02e79d268e4ccfa65d15eb6c929520'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.012114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b02e79d268e4ccfa65d15eb6c929520'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.042484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 842/1677] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:40 2022] DELETE /identity/v3/users/7b02e79d268e4ccfa65d15eb6c929520 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:41.047074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:41.048185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:41.048677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:41.088796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:41.090373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:41.118576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:41.121995 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:41.265234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 836/1678] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:41.265234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:41.265819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:41.268684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4aae3-6736-48c8-a2fa-7e3526aec33d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:41.287630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:41.289605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:41.290382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:41.290889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] PATH_INFO: `/v3/users/4e6fc6c2524c4d5d876960e3da581baa` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:41.293007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6fc6c2524c4d5d876960e3da581baa)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:41.293996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab662b-f732-4ca8-a2dd-7aa256ce1613 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:41.299568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:41.318223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:41.319878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:41.320402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:41.320826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] PATH_INFO: `/v3/projects/cbd0ccc961d3403d98288f0f62c9f36d` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:41.322793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd0ccc961d3403d98288f0f62c9f36d)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:41.324134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: WARNING py.warnings [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 15:50:41.324134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: warnings.warn(msg) Jun 30 15:50:41.324134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: Jun 30 15:50:41.325691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:41.340343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:41.451750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6fc6c2524c4d5d876960e3da581baa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.452886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6fc6c2524c4d5d876960e3da581baa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.488357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6fc6c2524c4d5d876960e3da581baa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.538877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6fc6c2524c4d5d876960e3da581baa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.539584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6fc6c2524c4d5d876960e3da581baa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.571446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 843/1679] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:41 2022] DELETE /identity/v3/users/4e6fc6c2524c4d5d876960e3da581baa => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:41.573910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:41.581540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:41.652702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:41.659707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd0ccc961d3403d98288f0f62c9f36d', 'request_id': 'req-342549d9-4094-494a-8f39-0b7866d56d32'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.693756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd0ccc961d3403d98288f0f62c9f36d', 'request_id': 'req-342549d9-4094-494a-8f39-0b7866d56d32'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.719520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd0ccc961d3403d98288f0f62c9f36d', 'request_id': 'req-342549d9-4094-494a-8f39-0b7866d56d32'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.742321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] The token cache is being invalidate because project cbd0ccc961d3403d98288f0f62c9f36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:41.742633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.743928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-342549d9-4094-494a-8f39-0b7866d56d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:41.746985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 837/1680] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:41 2022] DELETE /identity/v3/projects/cbd0ccc961d3403d98288f0f62c9f36d => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:41.752046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789346a9-b3a3-43bf-b35a-946c659188b9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:41.752969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789346a9-b3a3-43bf-b35a-946c659188b9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:41.753251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789346a9-b3a3-43bf-b35a-946c659188b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:41.757569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f7bdb-22eb-45df-946f-8db2f8571c33 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:41.794278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-789346a9-b3a3-43bf-b35a-946c659188b9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:41.795560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-789346a9-b3a3-43bf-b35a-946c659188b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:41.918443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:41.922348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:41.922640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:41.922920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:41.936319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:41.957499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 838/1681] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:41.962358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:41.962640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:41.962915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.009798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:42.012167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:42.037789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 839/1682] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:42.046849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.049102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:42.084824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd362-983a-48bb-97ea-468127f0111f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.095834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a604b2-9c5f-46e1-8fbc-dfcad7ad2e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:42.096540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a604b2-9c5f-46e1-8fbc-dfcad7ad2e08 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.096822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a604b2-9c5f-46e1-8fbc-dfcad7ad2e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.097498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a604b2-9c5f-46e1-8fbc-dfcad7ad2e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.128657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a604b2-9c5f-46e1-8fbc-dfcad7ad2e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.130151 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a604b2-9c5f-46e1-8fbc-dfcad7ad2e08 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.134613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.136165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.150754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:42.152922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 840/1683] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:42.155769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c151b2-ff6f-4250-8caa-0556e8060ff8 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.156658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c151b2-ff6f-4250-8caa-0556e8060ff8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.157511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c151b2-ff6f-4250-8caa-0556e8060ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.176643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 844/1684] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:42.204052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a5c151b2-ff6f-4250-8caa-0556e8060ff8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:42.206950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a5c151b2-ff6f-4250-8caa-0556e8060ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:42.216518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.218046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:42.242092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb671e8-6047-4ce7-9fda-acee4ba91d43 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.249743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 841/1685] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:42.251078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c151b2-ff6f-4250-8caa-0556e8060ff8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.273144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:42.273807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.274097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.274358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] PATH_INFO: `/v3/users/14edcf9cb9834f4f87049adb7c193c31` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.278906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14edcf9cb9834f4f87049adb7c193c31)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.282030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c151b2-ff6f-4250-8caa-0556e8060ff8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.284091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.303575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:42.304293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.304570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.304837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.305567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:42.305869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.307673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.364707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 842/1686] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:42.368090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.405907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb25e1-6a8c-485a-8a18-0bd15d868a94 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.426876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:42.427482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.427775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.431085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.431912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.441236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.443540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14edcf9cb9834f4f87049adb7c193c31'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:42.445248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14edcf9cb9834f4f87049adb7c193c31'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:42.489478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14edcf9cb9834f4f87049adb7c193c31'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:42.515973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14edcf9cb9834f4f87049adb7c193c31'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:42.517425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-c1d039ae-0eb0-47c1-8c5f-8658f9492d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14edcf9cb9834f4f87049adb7c193c31'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:42.527868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 843/1687] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:42 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:42.531730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.534272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:42.551819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 845/1688] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:42 2022] DELETE /identity/v3/users/14edcf9cb9834f4f87049adb7c193c31 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:50:42.554607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f720-f30b-4f80-89c8-dd63f70da670 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.554607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f720-f30b-4f80-89c8-dd63f70da670 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.554607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81f720-f30b-4f80-89c8-dd63f70da670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.585870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9b81f720-f30b-4f80-89c8-dd63f70da670 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:42.589547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9b81f720-f30b-4f80-89c8-dd63f70da670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:42.724826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 846/1689] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:42.729227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f900c-018b-48c6-aa33-9ffdb7e033e9 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.742299 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:42.743138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.743517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.743895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.744726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.751584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.772956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:42.791674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 844/1690] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:42 2022] POST /identity/v3/users => generated 285 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:42.797571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f720-f30b-4f80-89c8-dd63f70da670 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.815309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81f720-f30b-4f80-89c8-dd63f70da670 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.815704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.817616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:42.827019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:42.829425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.830373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.832266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.833395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.838210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.879720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 847/1691] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:42.883575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.907112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e963410-7910-459d-97c5-83939dcd2b8f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.920007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:42.920989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:42.921338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:42.921716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:42.922558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:42.926904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:42.953584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 848/1692] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:42.959154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:42.991814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9333e00-bfdc-4029-bf72-00a75a14c564 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:42.996591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13a72-89de-447f-bc02-687e58f952e5 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:43.004516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a281f00-2c01-422f-a1b9-972138319805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:43.005199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281f00-2c01-422f-a1b9-972138319805 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:43.005329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:43.005631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281f00-2c01-422f-a1b9-972138319805 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:43.006714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a281f00-2c01-422f-a1b9-972138319805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:43.007844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:43.010523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:43.012520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] PATH_INFO: `/v3/projects/83b804b30171463caf61b6a600b93a7c/users/ee26da93fa934de7949319c816e6d204/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:43.030408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a281f00-2c01-422f-a1b9-972138319805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:43.037877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a281f00-2c01-422f-a1b9-972138319805 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:43.062209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b804b30171463caf61b6a600b93a7c, user_id=ee26da93fa934de7949319c816e6d204, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:43.069928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:43.104424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 845/1693] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:50:43.127167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 849/1694] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:42 2022] PUT /identity/v3/projects/83b804b30171463caf61b6a600b93a7c/users/ee26da93fa934de7949319c816e6d204/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:43.129586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:43.130925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:43.131866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:43.224267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] MFA Rules not processed for user `ee26da93fa934de7949319c816e6d204`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:43.232802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:43.568546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 846/1695] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:43.574050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:43.657154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:43.803342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa93d9f-59de-4700-90d5-7b90d42941f0 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:43.812035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:43.813552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:43.836054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:43.836845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:43.837111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:43.837407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:43.838217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:43.850384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:43.860281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d714170-49e4-4ce6-a2dc-88c4f5a990e8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:43.882450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:43.884233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:43.884730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:43.884834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] PATH_INFO: `/v3/projects/121b500ff0bf4590ad411bc6101a9c12` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:43.887708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=121b500ff0bf4590ad411bc6101a9c12)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:43.889854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:43.934827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 850/1696] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:43 2022] POST /identity/v3/projects => generated 369 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:43.938965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:44.060621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121b500ff0bf4590ad411bc6101a9c12', 'request_id': 'req-865d8f9e-7991-4c0f-9164-934646db9cfd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.079080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121b500ff0bf4590ad411bc6101a9c12', 'request_id': 'req-865d8f9e-7991-4c0f-9164-934646db9cfd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.109193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121b500ff0bf4590ad411bc6101a9c12', 'request_id': 'req-865d8f9e-7991-4c0f-9164-934646db9cfd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.137845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] The token cache is being invalidate because project 121b500ff0bf4590ad411bc6101a9c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:44.138413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.140331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.149015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 847/1697] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:43 2022] DELETE /identity/v3/projects/121b500ff0bf4590ad411bc6101a9c12 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:44.153065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:44.153065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:44.155211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63b9dd-e72e-4e75-a27c-7365f0def31a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:44.204784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:44.355127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:44.355840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.356310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.356741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.357652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:44.359790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:44.370119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d8f9e-7991-4c0f-9164-934646db9cfd admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:44.391368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:44.392587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.393027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.393431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] PATH_INFO: `/v3/projects/ef2a47fe979b435a9352e23633aca709` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.396502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2a47fe979b435a9352e23633aca709)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:44.398628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:44.418945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 851/1698] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:43 2022] POST /identity/v3/users => generated 291 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:44.420079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:44.423484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:44.449616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48af037-5f10-4a2b-8318-fa8183c49f41 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:44.459504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:44.460769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.461357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.461712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] PATH_INFO: `/v3/users/2998758567cf48989dadb1cf2cbe3a48` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.463471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2998758567cf48989dadb1cf2cbe3a48)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:44.465879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:44.514475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2a47fe979b435a9352e23633aca709', 'request_id': 'req-e2072169-cf0d-4afe-8257-788690ee0ee7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.537110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2a47fe979b435a9352e23633aca709', 'request_id': 'req-e2072169-cf0d-4afe-8257-788690ee0ee7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.553564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2a47fe979b435a9352e23633aca709', 'request_id': 'req-e2072169-cf0d-4afe-8257-788690ee0ee7'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.568419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] The token cache is being invalidate because project ef2a47fe979b435a9352e23633aca709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:44.568883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.569754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e2072169-cf0d-4afe-8257-788690ee0ee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.572154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 848/1699] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:44 2022] DELETE /identity/v3/projects/ef2a47fe979b435a9352e23633aca709 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:44.573739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca7eb-3202-4087-a798-dbd6ad5722d2 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.573878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca7eb-3202-4087-a798-dbd6ad5722d2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.574170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ca7eb-3202-4087-a798-dbd6ad5722d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.586846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:44.598566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2998758567cf48989dadb1cf2cbe3a48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.600456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2998758567cf48989dadb1cf2cbe3a48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.613419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-170ca7eb-3202-4087-a798-dbd6ad5722d2 None None] MFA Rules not processed for user `2f3c67a263b14158a01fd906a31696cd`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:44.614669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-170ca7eb-3202-4087-a798-dbd6ad5722d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:44.624295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2998758567cf48989dadb1cf2cbe3a48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.651852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2998758567cf48989dadb1cf2cbe3a48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.652508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2998758567cf48989dadb1cf2cbe3a48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:44.680043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 852/1700] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:44 2022] DELETE /identity/v3/users/2998758567cf48989dadb1cf2cbe3a48 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:44.680708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:44.684223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:44.756541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 849/1701] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:44.778638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534001bf-d013-49c2-8776-ba0840014868 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.780084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534001bf-d013-49c2-8776-ba0840014868 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.781049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534001bf-d013-49c2-8776-ba0840014868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.809613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ec288-e162-48a4-accc-f7c84cf69705 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:44.832856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:44.834048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.834349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.834643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.835373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:44.837015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:44.841157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-534001bf-d013-49c2-8776-ba0840014868 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:44.843929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-534001bf-d013-49c2-8776-ba0840014868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:44.859247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 853/1702] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:44.861317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:44.869325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 850/1703] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:44.869325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534001bf-d013-49c2-8776-ba0840014868 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:44.871866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:44.898069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534001bf-d013-49c2-8776-ba0840014868 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:44.906640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ad98a-6ea4-41c3-8e95-96d167c0e6eb admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:44.910416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd557ad5-bdb8-41e5-aa13-e40e45d7125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:44.914113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd557ad5-bdb8-41e5-aa13-e40e45d7125f admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.915072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd557ad5-bdb8-41e5-aa13-e40e45d7125f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.915366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd557ad5-bdb8-41e5-aa13-e40e45d7125f admin admin] PATH_INFO: `/v3/projects/c7d8d3258e31434d918dbe6f87286269/users/a66b14c20ff2427ab25ec24096c4317c/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.921793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:44.923173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:44.923632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:44.924126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] PATH_INFO: `/v3/projects/e59bf7d520e340948ec3cc41bb0598a2` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:44.926016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e59bf7d520e340948ec3cc41bb0598a2)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:44.928257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:44.957314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd557ad5-bdb8-41e5-aa13-e40e45d7125f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d8d3258e31434d918dbe6f87286269, user_id=a66b14c20ff2427ab25ec24096c4317c, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:44.963193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd557ad5-bdb8-41e5-aa13-e40e45d7125f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:44.999094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 851/1704] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:44 2022] PUT /identity/v3/projects/c7d8d3258e31434d918dbe6f87286269/users/a66b14c20ff2427ab25ec24096c4317c/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:45.000578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.000867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.001144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.018652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e59bf7d520e340948ec3cc41bb0598a2', 'request_id': 'req-e5792c96-ad18-4ff8-a65e-de678d8fa31f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.042362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e59bf7d520e340948ec3cc41bb0598a2', 'request_id': 'req-e5792c96-ad18-4ff8-a65e-de678d8fa31f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.050415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:45.050958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.053262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e59bf7d520e340948ec3cc41bb0598a2', 'request_id': 'req-e5792c96-ad18-4ff8-a65e-de678d8fa31f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.070286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] The token cache is being invalidate because project e59bf7d520e340948ec3cc41bb0598a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:45.071244 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.072405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.077763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 854/1705] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:44 2022] DELETE /identity/v3/projects/e59bf7d520e340948ec3cc41bb0598a2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:45.077763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:45.079425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.080284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:45.153816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:45.176976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 852/1706] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:45.181359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:45.184810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.190329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5792c96-ad18-4ff8-a65e-de678d8fa31f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:45.202757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:45.203459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.203742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.204038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.204875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:45.216262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:45.216347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a5efb-22c8-4dbd-8c1a-19b67b6c1e0c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:45.225448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:45.226352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.226631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.226907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] PATH_INFO: `/v3/users/4a8d4f65d2b24e4c96d8a9d1d21300d1` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.228386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8d4f65d2b24e4c96d8a9d1d21300d1)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:45.230682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:45.233388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 855/1707] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:45.233733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:45.255835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a307588-4558-41a0-90a2-da308d2b2835 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:45.272858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c7189-bdcd-4993-a0ca-c60b14b020aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:45.273563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7189-bdcd-4993-a0ca-c60b14b020aa admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.273852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7189-bdcd-4993-a0ca-c60b14b020aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.274113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c7189-bdcd-4993-a0ca-c60b14b020aa admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.274864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c7189-bdcd-4993-a0ca-c60b14b020aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:45.275172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c7189-bdcd-4993-a0ca-c60b14b020aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:45.278348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c7189-bdcd-4993-a0ca-c60b14b020aa admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:45.313824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 856/1708] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:45.313824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.313824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.313824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.344773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8d4f65d2b24e4c96d8a9d1d21300d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.361926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8d4f65d2b24e4c96d8a9d1d21300d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.400633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8d4f65d2b24e4c96d8a9d1d21300d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.401463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8d4f65d2b24e4c96d8a9d1d21300d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.402218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8d4f65d2b24e4c96d8a9d1d21300d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:45.421526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] MFA Rules not processed for user `ead6569ad30b496092f49b3fe0c8a706`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:45.423342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.436102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 853/1709] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:45 2022] DELETE /identity/v3/users/4a8d4f65d2b24e4c96d8a9d1d21300d1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:45.437256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:45.438549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.603042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 857/1710] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:45.603913 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:45.611493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.649891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09606728-2088-4f43-9a5e-0056b915091b admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:45.657597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa67235-39f8-47d2-afbc-14c6919f1580 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:45.666511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b425aff-9008-40f0-8a9b-a76c2b2b5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:45.668119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b425aff-9008-40f0-8a9b-a76c2b2b5fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.668516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b425aff-9008-40f0-8a9b-a76c2b2b5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.668933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b425aff-9008-40f0-8a9b-a76c2b2b5fa6 admin admin] PATH_INFO: `/v3/projects/c7d8d3258e31434d918dbe6f87286269/users/a66b14c20ff2427ab25ec24096c4317c/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.670218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e98e4-59ff-41d7-b715-6f23a783d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:45.670860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e98e4-59ff-41d7-b715-6f23a783d43e admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.671145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e98e4-59ff-41d7-b715-6f23a783d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.671415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e98e4-59ff-41d7-b715-6f23a783d43e admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.672153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e98e4-59ff-41d7-b715-6f23a783d43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:45.674086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e98e4-59ff-41d7-b715-6f23a783d43e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:45.677992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b425aff-9008-40f0-8a9b-a76c2b2b5fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d8d3258e31434d918dbe6f87286269, user_id=a66b14c20ff2427ab25ec24096c4317c, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:45.680785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b425aff-9008-40f0-8a9b-a76c2b2b5fa6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:45.705771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 854/1711] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:45 2022] PUT /identity/v3/projects/c7d8d3258e31434d918dbe6f87286269/users/a66b14c20ff2427ab25ec24096c4317c/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:45.707524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee15f5-9d23-43b2-9ebb-389789944230 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.707827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee15f5-9d23-43b2-9ebb-389789944230 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.708482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee15f5-9d23-43b2-9ebb-389789944230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.754844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 858/1712] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:45 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:45.761054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:45.762695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:45.763882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:45.783880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-5bee15f5-9d23-43b2-9ebb-389789944230 None None] MFA Rules not processed for user `ead6569ad30b496092f49b3fe0c8a706`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:45.787424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5bee15f5-9d23-43b2-9ebb-389789944230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.809102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] MFA Rules not processed for user `a66b14c20ff2427ab25ec24096c4317c`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:45.811303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:45.848635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:45.958560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 855/1713] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:45.958560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee15f5-9d23-43b2-9ebb-389789944230 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:46.005149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 859/1714] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:45 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:46.005149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:46.019779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:46.066524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee15f5-9d23-43b2-9ebb-389789944230 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:46.074438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:46.080317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:46.081030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.081314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.081634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.082392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:46.084904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:46.130283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 856/1715] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:45 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:46.132101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:46.137787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:46.261461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d12150-9a7d-496a-86d9-981aa652a47d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:46.265522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cc73-191c-479c-80b7-87d4e6a914ae admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:46.274567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:46.275267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.275568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.275858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.279924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:46.280874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.281351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.281820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.294400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:46.295982 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:46.303525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:46.307307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:46.308284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 857/1716] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:46.309216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:46.320618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ddae1c-43c3-4e00-bb29-b1797fd1c170 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:46.327083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:46.327839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.328219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.328563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.329361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:46.331203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:46.335449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 860/1717] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:46.355695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 858/1718] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:46.359399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:46.394005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70346c44-f9da-4196-9ca0-788bde8b392c service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:46.426979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168da5b7-7a2a-41e7-b134-19dcc310c480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:46.437492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168da5b7-7a2a-41e7-b134-19dcc310c480 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.437492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168da5b7-7a2a-41e7-b134-19dcc310c480 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.437492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168da5b7-7a2a-41e7-b134-19dcc310c480 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.437492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168da5b7-7a2a-41e7-b134-19dcc310c480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:46.438161 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168da5b7-7a2a-41e7-b134-19dcc310c480 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:46.473562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 861/1719] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:46.474309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:46.494759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290832d-05c4-49b5-9c38-e68194975c2d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:46.507135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:46.508069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.510753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.511058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] PATH_INFO: `/v3/projects/4a50129ee95f42dda024fecff91547c1/users/a3519b4e556e416b8e78ce28a830cc99/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.549774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a50129ee95f42dda024fecff91547c1, user_id=a3519b4e556e416b8e78ce28a830cc99, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:46.553106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:46.603308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 859/1720] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:46 2022] PUT /identity/v3/projects/4a50129ee95f42dda024fecff91547c1/users/a3519b4e556e416b8e78ce28a830cc99/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:50:46.617988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0728-a080-43b7-ad59-fda00e6a8205 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.618279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0728-a080-43b7-ad59-fda00e6a8205 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.618556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a0728-a080-43b7-ad59-fda00e6a8205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.664636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3f0a0728-a080-43b7-ad59-fda00e6a8205 None None] MFA Rules not processed for user `a3519b4e556e416b8e78ce28a830cc99`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:46.665875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3f0a0728-a080-43b7-ad59-fda00e6a8205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:46.673005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:46.675964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:46.777156 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:46.801260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 862/1721] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:46.871326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234450f4-3497-48ae-829e-36edb9a7da11 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:46.880795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:46.882408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.882408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.897101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] PATH_INFO: `/v3/projects/56583df03be449cab79fda351ebeb904` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.900381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=56583df03be449cab79fda351ebeb904)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:46.902344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:46.982371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15a6b2-8176-4364-a6ca-5e45baa530b0 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.983018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15a6b2-8176-4364-a6ca-5e45baa530b0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.983534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15a6b2-8176-4364-a6ca-5e45baa530b0 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:46.985314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 863/1722] 10.222.0.50 () {54 vars in 912 bytes} [Thu Jun 30 15:50:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:50:46.993603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c493661-e302-4e41-908f-1f27a2eeae79 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:46.994006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c493661-e302-4e41-908f-1f27a2eeae79 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:46.994390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c493661-e302-4e41-908f-1f27a2eeae79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.039406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56583df03be449cab79fda351ebeb904', 'request_id': 'req-efceacd0-d807-4c1a-875b-1388094069bd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.063821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56583df03be449cab79fda351ebeb904', 'request_id': 'req-efceacd0-d807-4c1a-875b-1388094069bd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.093494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3c493661-e302-4e41-908f-1f27a2eeae79 None None] MFA Rules not processed for user `5e17d25a83fe48d99ddbd209fbea8486`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:47.098818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56583df03be449cab79fda351ebeb904', 'request_id': 'req-efceacd0-d807-4c1a-875b-1388094069bd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.107890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3c493661-e302-4e41-908f-1f27a2eeae79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.125637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] The token cache is being invalidate because project 56583df03be449cab79fda351ebeb904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:47.126232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.128165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.143540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 860/1723] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:46 2022] DELETE /identity/v3/projects/56583df03be449cab79fda351ebeb904 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:47.143540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:47.145397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.306303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efceacd0-d807-4c1a-875b-1388094069bd admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:47.307182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 864/1724] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:50:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:47.309087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4e53-bd1b-4f0f-953b-db691652b64e None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.310588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4e53-bd1b-4f0f-953b-db691652b64e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.311081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f4e53-bd1b-4f0f-953b-db691652b64e None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.312279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:47.313842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 865/1725] 10.222.0.50 () {54 vars in 912 bytes} [Thu Jun 30 15:50:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:50:47.315035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411256-4e45-4663-8ec3-0a2357945018 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.315320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411256-4e45-4663-8ec3-0a2357945018 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.315596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c411256-4e45-4663-8ec3-0a2357945018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.321805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.322427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.322904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] PATH_INFO: `/v3/projects/900afaf15f8d4adea6f0a8bf256105a5` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.324844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=900afaf15f8d4adea6f0a8bf256105a5)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:47.328326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:47.363246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5c411256-4e45-4663-8ec3-0a2357945018 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:47.365475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5c411256-4e45-4663-8ec3-0a2357945018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.384918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 866/1726] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:47.390585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0956a5-0b5c-4e7b-80f0-7f11fb0296f8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.391139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0956a5-0b5c-4e7b-80f0-7f11fb0296f8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.392085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0956a5-0b5c-4e7b-80f0-7f11fb0296f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.430057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:47.433649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900afaf15f8d4adea6f0a8bf256105a5', 'request_id': 'req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.441655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4f0956a5-0b5c-4e7b-80f0-7f11fb0296f8 None None] MFA Rules not processed for user `5e17d25a83fe48d99ddbd209fbea8486`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:47.445701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4f0956a5-0b5c-4e7b-80f0-7f11fb0296f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.445840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900afaf15f8d4adea6f0a8bf256105a5', 'request_id': 'req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.464593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900afaf15f8d4adea6f0a8bf256105a5', 'request_id': 'req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.468050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:47.503941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] The token cache is being invalidate because project 900afaf15f8d4adea6f0a8bf256105a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:47.504326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.505122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:47.513838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 861/1727] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:47 2022] DELETE /identity/v3/projects/900afaf15f8d4adea6f0a8bf256105a5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:47.513838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:47.516773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.602776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 867/1728] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:47.606599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96182ce4-e847-4158-b824-c235f749642f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.607055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96182ce4-e847-4158-b824-c235f749642f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.607408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96182ce4-e847-4158-b824-c235f749642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.657056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-96182ce4-e847-4158-b824-c235f749642f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:47.658325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-96182ce4-e847-4158-b824-c235f749642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.668224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0421a-dc1b-4e2a-aa62-f6111c7b8ca8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:47.678008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29167039-7df9-40dd-b3ef-a9774662ee5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:47.678685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167039-7df9-40dd-b3ef-a9774662ee5d service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.680799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167039-7df9-40dd-b3ef-a9774662ee5d service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.681104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167039-7df9-40dd-b3ef-a9774662ee5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.684029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-29167039-7df9-40dd-b3ef-a9774662ee5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.714880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29167039-7df9-40dd-b3ef-a9774662ee5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:47.716504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29167039-7df9-40dd-b3ef-a9774662ee5d service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:47.757409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 862/1729] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:47.759496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9406f-51c8-43f7-b58c-0cc6b40f4716 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.762174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9406f-51c8-43f7-b58c-0cc6b40f4716 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.762932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9406f-51c8-43f7-b58c-0cc6b40f4716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.814432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-15f9406f-51c8-43f7-b58c-0cc6b40f4716 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:47.816676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-15f9406f-51c8-43f7-b58c-0cc6b40f4716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:47.869212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 868/1730] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:47.876420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96182ce4-e847-4158-b824-c235f749642f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:47.904860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96182ce4-e847-4158-b824-c235f749642f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:47.930845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fe5ad-a0eb-48b2-a0f0-50d90794686c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:47.931527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe5ad-a0eb-48b2-a0f0-50d90794686c service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:47.931805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe5ad-a0eb-48b2-a0f0-50d90794686c service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:47.932102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fe5ad-a0eb-48b2-a0f0-50d90794686c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:47.974859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fe5ad-a0eb-48b2-a0f0-50d90794686c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:47.979618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fe5ad-a0eb-48b2-a0f0-50d90794686c service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:48.021457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 869/1731] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:48.025866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a76f2-1e4c-496e-9974-ea3abef2c6b8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.026162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a76f2-1e4c-496e-9974-ea3abef2c6b8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.026445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a76f2-1e4c-496e-9974-ea3abef2c6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.095295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-2c7a76f2-1e4c-496e-9974-ea3abef2c6b8 None None] MFA Rules not processed for user `7f2f23e763704ce2b0969e24ffde9243`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:48.108629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2c7a76f2-1e4c-496e-9974-ea3abef2c6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:48.113224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 863/1732] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:48.113224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9406f-51c8-43f7-b58c-0cc6b40f4716 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:48.150078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9406f-51c8-43f7-b58c-0cc6b40f4716 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:48.176173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76850335-08bd-466b-86d3-721085a415b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:48.176915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76850335-08bd-466b-86d3-721085a415b6 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.180792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76850335-08bd-466b-86d3-721085a415b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.181131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76850335-08bd-466b-86d3-721085a415b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.181913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76850335-08bd-466b-86d3-721085a415b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:48.182223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76850335-08bd-466b-86d3-721085a415b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:48.187000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76850335-08bd-466b-86d3-721085a415b6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:48.234239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 864/1733] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:48.240966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed84267-8198-417c-b921-2466c575c267 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.241583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed84267-8198-417c-b921-2466c575c267 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.242682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed84267-8198-417c-b921-2466c575c267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.293180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 870/1734] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:48.297742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a76f2-1e4c-496e-9974-ea3abef2c6b8 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:48.320998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-bed84267-8198-417c-b921-2466c575c267 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:48.323816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bed84267-8198-417c-b921-2466c575c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:48.336354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 865/1735] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:48.338210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a76f2-1e4c-496e-9974-ea3abef2c6b8 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:48.338738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed84267-8198-417c-b921-2466c575c267 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:48.359528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed84267-8198-417c-b921-2466c575c267 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:48.361495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807a2e7-5b1e-433c-be1b-dc14b034c0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:48.367255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807a2e7-5b1e-433c-be1b-dc14b034c0bb admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.369261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807a2e7-5b1e-433c-be1b-dc14b034c0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.370590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807a2e7-5b1e-433c-be1b-dc14b034c0bb admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.371774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:48.373782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807a2e7-5b1e-433c-be1b-dc14b034c0bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:48.375147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.376373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.376930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.377931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:48.381641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:48.386987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807a2e7-5b1e-433c-be1b-dc14b034c0bb admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:48.389521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:48.449054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 866/1736] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:48.453524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:48.502402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa52ea-b573-4c3b-a51a-1dc7326317f6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:48.506082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 871/1737] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/projects => generated 377 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:48.510261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb48fb8-6bb1-4c5e-b4f2-841716009ac6 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.510594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb48fb8-6bb1-4c5e-b4f2-841716009ac6 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.510822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb48fb8-6bb1-4c5e-b4f2-841716009ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.563760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:48.596669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ffb48fb8-6bb1-4c5e-b4f2-841716009ac6 None None] MFA Rules not processed for user `d30ffdb0374c4600a09476cfb3a07cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:48.604416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ffb48fb8-6bb1-4c5e-b4f2-841716009ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:48.671211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:48.673180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.673469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.674984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.675716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:48.682979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:48.695368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:48.753070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 867/1738] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/projects => generated 363 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:48.753965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:48.788389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 872/1739] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:48.790603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb48fb8-6bb1-4c5e-b4f2-841716009ac6 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:48.814976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb48fb8-6bb1-4c5e-b4f2-841716009ac6 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:48.882074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed88f96-9cfc-4621-9cde-a81dd8ebc11f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:48.891760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a53fec-c826-4c1a-a647-6241c6394294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:48.892776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a53fec-c826-4c1a-a647-6241c6394294 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.893131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a53fec-c826-4c1a-a647-6241c6394294 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.893668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a53fec-c826-4c1a-a647-6241c6394294 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.894463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a53fec-c826-4c1a-a647-6241c6394294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:48.899019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a53fec-c826-4c1a-a647-6241c6394294 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:48.960624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 868/1740] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:48.965441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6cdc-3415-4013-ae3d-df81a10a242e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.970045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6cdc-3415-4013-ae3d-df81a10a242e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.970379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b6cdc-3415-4013-ae3d-df81a10a242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.983034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:48.983982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:48.984357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:48.984658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:48.986203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:48.994755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.027227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-708b6cdc-3415-4013-ae3d-df81a10a242e None None] MFA Rules not processed for user `d30ffdb0374c4600a09476cfb3a07cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:49.029276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-708b6cdc-3415-4013-ae3d-df81a10a242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.062808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 873/1741] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/users => generated 282 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:49.064039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.070160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.161207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 869/1742] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:49.163048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6cdc-3415-4013-ae3d-df81a10a242e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.195282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b6cdc-3415-4013-ae3d-df81a10a242e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.210116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.212252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.213688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.213970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.215287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.219337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.251445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 870/1743] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:49.255886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.318971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024dd746-7dd6-4a03-99c6-622119c429a8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.328837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.330339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.330819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.331305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.332179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ce16e-5e2b-4b34-aa67-6121c6138526 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.333417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.336255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.339084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.339824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.340138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.340415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.342321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.355618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 871/1744] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:49.357437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.361579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.383377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:49.435136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.437790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.463258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 874/1745] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:49.464256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.466062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.485622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:49.503400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6151c1b-79f7-48dd-9494-55b5afb866b0 service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.511298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7bd533-414a-4a43-b98d-7a2fecd6793a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.511550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.512252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.512537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.512813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.521814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.522843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.524450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.524830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.530655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.564673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.567745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.592181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 872/1746] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:49.598468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.622119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfaa920-bde5-477e-9e4f-6943f1f37491 service placement] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.637471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.638383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.639179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.639565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.639901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.640734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.642419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.646614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.669541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 873/1747] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:49.670542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.678073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.697006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 875/1748] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:49.700758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.704859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:49.709034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69f630-c036-43e8-9a47-780690a88194 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.723451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.724256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.724594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.725069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.743613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f986614-73d4-45e2-b22c-00194ebe921f service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.744750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.746373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.756781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.757682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.757965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.758237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] PATH_INFO: `/v3/users/ee26da93fa934de7949319c816e6d204` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.760083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee26da93fa934de7949319c816e6d204)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.762039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.771517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 874/1749] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:49.774126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.817087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9c3f4-a0d4-4ee5-b82a-2d0136e5fb7f service placement] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.833751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.834449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.834725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.834993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.835731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.838730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:49.901006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 875/1750] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:49.905107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:49.933831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433b945-ed37-4ba1-9685-826e7930e215 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:49.941389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:49.942303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:49.942582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:49.942852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] PATH_INFO: `/v3/projects/fe494de3259b44b1bc8c1265d919a21a/users/2a43fb1726c740d18918e3ca1835b036/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:49.994378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe494de3259b44b1bc8c1265d919a21a, user_id=2a43fb1726c740d18918e3ca1835b036, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:49.999363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:50.058828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 876/1751] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:49 2022] PUT /identity/v3/projects/fe494de3259b44b1bc8c1265d919a21a/users/2a43fb1726c740d18918e3ca1835b036/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:50.058828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:50.351145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee26da93fa934de7949319c816e6d204'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:50.351636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d13450-621e-4a8a-b86a-327a9ef14e4f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:50.354977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee26da93fa934de7949319c816e6d204'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:50.371129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1aac6d-52d0-4b80-babe-6fc1ac08b2e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:50.372099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aac6d-52d0-4b80-babe-6fc1ac08b2e8 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:50.372662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aac6d-52d0-4b80-babe-6fc1ac08b2e8 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:50.373196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1aac6d-52d0-4b80-babe-6fc1ac08b2e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:50.375368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2d1aac6d-52d0-4b80-babe-6fc1ac08b2e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:50.444640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee26da93fa934de7949319c816e6d204'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:50.479419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee26da93fa934de7949319c816e6d204'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:50.480670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee26da93fa934de7949319c816e6d204'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:50.518054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 876/1752] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:49 2022] DELETE /identity/v3/users/ee26da93fa934de7949319c816e6d204 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:50.518959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:50.526681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:50.640719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aac6d-52d0-4b80-babe-6fc1ac08b2e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:50.647980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:50.667153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1aac6d-52d0-4b80-babe-6fc1ac08b2e8 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:50.712770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 877/1753] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:50.714321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaa811-9733-4da8-a093-793361e51cfa None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:50.714609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaa811-9733-4da8-a093-793361e51cfa None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:50.714883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaa811-9733-4da8-a093-793361e51cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:50.722018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf553e85-558e-4664-8727-ed680ac2f9a7 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:50.739460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:50.745235 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:50.745846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:50.746868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] PATH_INFO: `/v3/projects/7d6934f696f84c2d88ed56193ddecada/users/d0354711887c4ce3bd96d65692702fbb/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:50.797224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6bfaa811-9733-4da8-a093-793361e51cfa None None] MFA Rules not processed for user `2a43fb1726c740d18918e3ca1835b036`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:50.798520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6bfaa811-9733-4da8-a093-793361e51cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:50.803805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6934f696f84c2d88ed56193ddecada, user_id=d0354711887c4ce3bd96d65692702fbb, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:50.829723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:50.837945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:50.863587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 877/1754] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:50 2022] PUT /identity/v3/projects/7d6934f696f84c2d88ed56193ddecada/users/d0354711887c4ce3bd96d65692702fbb/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:50.864913 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:50.878276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.027953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 878/1755] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:51.031421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceed119-6aea-4543-9ab4-013d62ad623a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:51.031828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceed119-6aea-4543-9ab4-013d62ad623a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:51.033685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceed119-6aea-4543-9ab4-013d62ad623a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:51.073585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-eceed119-6aea-4543-9ab4-013d62ad623a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:51.075358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-eceed119-6aea-4543-9ab4-013d62ad623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.169342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 879/1756] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:51.171118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:51.171502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:51.172355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:51.211874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] MFA Rules not processed for user `d0354711887c4ce3bd96d65692702fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:51.215176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.254107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ead6e-fecf-445c-9f7e-250d4a8797ce admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:51.270083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:51.275182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:51.275479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:51.275774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] PATH_INFO: `/v3/users/a3519b4e556e416b8e78ce28a830cc99` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:51.281922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3519b4e556e416b8e78ce28a830cc99)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:51.284241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:51.399561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 880/1757] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:51.401304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:51.407397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.439346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3519b4e556e416b8e78ce28a830cc99'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:51.440184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3519b4e556e416b8e78ce28a830cc99'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:51.450389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10427eda-6906-4800-b2da-bde333aa0c3c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:51.472352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3519b4e556e416b8e78ce28a830cc99'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:51.483731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:51.484453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:51.484741 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:51.485012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:51.485844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:51.498169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:51.505531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3519b4e556e416b8e78ce28a830cc99'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:51.506174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3519b4e556e416b8e78ce28a830cc99'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:51.551616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 878/1758] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:50 2022] DELETE /identity/v3/users/a3519b4e556e416b8e78ce28a830cc99 => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:51.556118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:51.560220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.565099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 881/1759] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:51 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:51.567306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:51.575638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.662350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:51.720445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:51.745147 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24d723-d74b-4cc9-bcdf-3afc2833a53d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:51.780599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ca421-7e4c-4661-a1e9-272782c70bad admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:51.795749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba56d883-a1b0-4eeb-a987-502e981e1600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:51.796487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56d883-a1b0-4eeb-a987-502e981e1600 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:51.796772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56d883-a1b0-4eeb-a987-502e981e1600 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:51.797049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba56d883-a1b0-4eeb-a987-502e981e1600 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:51.797943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56d883-a1b0-4eeb-a987-502e981e1600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:51.801527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba56d883-a1b0-4eeb-a987-502e981e1600 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:51.857871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:51.858839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:51.859280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:51.860847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:51.864408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 882/1760] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:51 2022] POST /identity/v3/projects => generated 363 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:51.864408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7930d-c301-4e49-b12a-2de82bceb311 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:51.864408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7930d-c301-4e49-b12a-2de82bceb311 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:51.864408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7930d-c301-4e49-b12a-2de82bceb311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:51.869030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.896445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:51.898396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-0ee7930d-c301-4e49-b12a-2de82bceb311 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:51.899667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-0ee7930d-c301-4e49-b12a-2de82bceb311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.900900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:51.926200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 879/1761] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:51.927415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:51.933606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:51.956399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58418c62-19b0-4885-934a-edbc6adaaad0 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:51.997735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 883/1762] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:51.997735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7930d-c301-4e49-b12a-2de82bceb311 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.008849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7930d-c301-4e49-b12a-2de82bceb311 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.014785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.016056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.016056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.016250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.017164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.023181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.054603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 884/1763] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:51 2022] POST /identity/v3/users => generated 288 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:52.056543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.059078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:52.138788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.139726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.140471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.140471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.141239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.143559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.181836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4581c9-3c34-4a1f-8d39-c26546c4f311 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.184959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 880/1764] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:51 2022] POST /identity/v3/users => generated 295 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:52.186813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.190305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.193116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.193573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.194844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.199618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50589b0f-3312-4e74-af05-b2a95e952ecb admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.205301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.206273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.206781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.207139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.208146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.209972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.217314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.220557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.224014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 881/1765] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:52.225130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.238337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf13f6-6e83-432f-b17e-3ad0e9a7a2ac admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.244569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.245355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.245751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.246096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.247098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.249427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 885/1766] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:52.249835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.250680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.281364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddabb0-1c49-4fee-aa10-833c9c397942 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.290954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 882/1767] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:52.291574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.292483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.292870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.293369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] PATH_INFO: `/v3/projects/bc9753f32875428a87c4cef8007e4235/users/c1ee9c16aadd4ac5a753f44fd0a70483/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.295805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.328281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7691298-74e0-49cc-b376-3fa3019c7c43 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.330822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9753f32875428a87c4cef8007e4235, user_id=c1ee9c16aadd4ac5a753f44fd0a70483, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.333857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.345154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.347553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.350272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.351227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] PATH_INFO: `/v3/projects/ab7ff8e8b6774a1484c61bef5208c4bb/users/df6d6f5f724443028d68062bc16a4991/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.369172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 886/1768] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:52 2022] PUT /identity/v3/projects/bc9753f32875428a87c4cef8007e4235/users/c1ee9c16aadd4ac5a753f44fd0a70483/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:52.372791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.413864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7ff8e8b6774a1484c61bef5208c4bb, user_id=df6d6f5f724443028d68062bc16a4991, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.419131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:52.425615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.480783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 883/1769] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:52 2022] PUT /identity/v3/projects/ab7ff8e8b6774a1484c61bef5208c4bb/users/df6d6f5f724443028d68062bc16a4991/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:52.484611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.611016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:52.665203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574378b-3887-4fb7-95e5-6cad96b9fccc admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.675170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.675870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.676222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.676513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.677234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.681972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.698751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1474e6a-4ac2-4251-af3a-fa98ee7c6881 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.722840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 884/1770] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:52.724978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.729372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.730581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.732159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.737056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.738515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.741773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.746988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91553ff-0886-4354-8ee7-cfa7a4eb26a1 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.757373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217af363-30ad-461f-b493-1444fcc89239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:52.758295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af363-30ad-461f-b493-1444fcc89239 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.758603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af363-30ad-461f-b493-1444fcc89239 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.762230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217af363-30ad-461f-b493-1444fcc89239 admin admin] PATH_INFO: `/v3/projects/ab7ff8e8b6774a1484c61bef5208c4bb/users/df6d6f5f724443028d68062bc16a4991/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.766568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217af363-30ad-461f-b493-1444fcc89239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7ff8e8b6774a1484c61bef5208c4bb, user_id=df6d6f5f724443028d68062bc16a4991, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:52.769157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217af363-30ad-461f-b493-1444fcc89239 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:52.794637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 887/1771] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:52.796815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:52.805958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 885/1772] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:52 2022] PUT /identity/v3/projects/ab7ff8e8b6774a1484c61bef5208c4bb/users/df6d6f5f724443028d68062bc16a4991/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:52.815876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1821c-49d2-4f11-b157-a79e4e647e5e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:52.816534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c55c84-290a-49f5-8e0c-2bfb912a714a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:52.816690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1821c-49d2-4f11-b157-a79e4e647e5e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:52.817081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1821c-49d2-4f11-b157-a79e4e647e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:52.887662 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f7e1821c-49d2-4f11-b157-a79e4e647e5e None None] MFA Rules not processed for user `df6d6f5f724443028d68062bc16a4991`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:52.892224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f7e1821c-49d2-4f11-b157-a79e4e647e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:53.064901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ed1a71-f7f3-46e1-b539-432a07614ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:53.069466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed1a71-f7f3-46e1-b539-432a07614ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:53.074293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed1a71-f7f3-46e1-b539-432a07614ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:53.074994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed1a71-f7f3-46e1-b539-432a07614ef0 admin admin] PATH_INFO: `/v3/projects/bc9753f32875428a87c4cef8007e4235/users/c1ee9c16aadd4ac5a753f44fd0a70483/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:53.084209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed1a71-f7f3-46e1-b539-432a07614ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc9753f32875428a87c4cef8007e4235, user_id=c1ee9c16aadd4ac5a753f44fd0a70483, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:53.099416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ed1a71-f7f3-46e1-b539-432a07614ef0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:53.173765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 886/1773] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:53.177043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:53.177345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:53.180167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:53.198679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 888/1774] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:52 2022] PUT /identity/v3/projects/bc9753f32875428a87c4cef8007e4235/users/c1ee9c16aadd4ac5a753f44fd0a70483/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:53.216048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:53.216722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:53.217291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:53.225501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] MFA Rules not processed for user `d30ffdb0374c4600a09476cfb3a07cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:53.251947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:53.368843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] MFA Rules not processed for user `c1ee9c16aadd4ac5a753f44fd0a70483`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:53.370970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:53.430392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 887/1775] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:53.435531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:53.441833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:53.686810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 889/1776] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:53.707540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:53.709528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:53.726043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d1155-e2a0-448a-bc2f-fad3dc011786 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:53.746677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:53.747893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:53.749969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:53.750503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:53.755599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a1a0b-bbf5-4005-b72b-0f4036e35517 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:53.761799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:53.768383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565dd53-b7b4-495b-a3e8-b3639c24fea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:53.771220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565dd53-b7b4-495b-a3e8-b3639c24fea7 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:53.771848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565dd53-b7b4-495b-a3e8-b3639c24fea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:53.773334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565dd53-b7b4-495b-a3e8-b3639c24fea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:53.800858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:53.803264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565dd53-b7b4-495b-a3e8-b3639c24fea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:53.807905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565dd53-b7b4-495b-a3e8-b3639c24fea7 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:53.849053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 890/1777] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:53.961168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:53.961861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:53.962335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:54.017329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:54.020972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:54.035085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:54.051824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:54.053556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:54.083879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 888/1778] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:54.085222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:54.109438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da855db-3468-49ba-ba29-cd62bbc244ef service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:54.118487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f301de8-4f39-49c1-abe7-d9dd5f42edf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:54.119739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f301de8-4f39-49c1-abe7-d9dd5f42edf0 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:54.121143 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f301de8-4f39-49c1-abe7-d9dd5f42edf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:54.122321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f301de8-4f39-49c1-abe7-d9dd5f42edf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:54.128725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5f301de8-4f39-49c1-abe7-d9dd5f42edf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:54.264785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f301de8-4f39-49c1-abe7-d9dd5f42edf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:54.268703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f301de8-4f39-49c1-abe7-d9dd5f42edf0 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:54.272471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 891/1779] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:50:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:54.302524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 889/1780] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:54 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:54.628772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:54.631709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:54.888675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93fa0f-d59d-4146-ba23-5e84ee1dc69c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:54.909011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:54.910340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:54.910854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:54.911631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:54.938917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:54.940764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:54.968327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 892/1781] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:50:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:50:55.333308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e2198-f2a5-451d-8141-001f4166d20f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:55.333842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e2198-f2a5-451d-8141-001f4166d20f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:55.333916 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e2198-f2a5-451d-8141-001f4166d20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:55.416934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0b8e2198-f2a5-451d-8141-001f4166d20f None None] MFA Rules not processed for user `7f2f23e763704ce2b0969e24ffde9243`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:55.418107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0b8e2198-f2a5-451d-8141-001f4166d20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:55.433762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:55.435856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:55.606470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d016c-6849-41b3-addc-04ee448c7d2d service swift] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:55.618507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 890/1782] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:55.623077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:55.626212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:55.626212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:55.627225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] PATH_INFO: `/v3/projects/83b804b30171463caf61b6a600b93a7c` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:55.628961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b804b30171463caf61b6a600b93a7c)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:55.631330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:55.700666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:55.702960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b804b30171463caf61b6a600b93a7c', 'request_id': 'req-6f7463a3-26a0-407b-aaa8-ffe41c6855da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:55.714814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b804b30171463caf61b6a600b93a7c', 'request_id': 'req-6f7463a3-26a0-407b-aaa8-ffe41c6855da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:55.728450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b804b30171463caf61b6a600b93a7c', 'request_id': 'req-6f7463a3-26a0-407b-aaa8-ffe41c6855da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:55.745958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] The token cache is being invalidate because project 83b804b30171463caf61b6a600b93a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:55.746644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:55.747552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:55.750038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 893/1783] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:55 2022] DELETE /identity/v3/projects/83b804b30171463caf61b6a600b93a7c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:55.755644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e2198-f2a5-451d-8141-001f4166d20f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:55.773665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e2198-f2a5-451d-8141-001f4166d20f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:55.886671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:55.908556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:55.910403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:55.912284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:55.912615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] PATH_INFO: `/v3/users/a66b14c20ff2427ab25ec24096c4317c` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:55.915836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66b14c20ff2427ab25ec24096c4317c)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:55.922827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:56.066950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66b14c20ff2427ab25ec24096c4317c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:56.066950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66b14c20ff2427ab25ec24096c4317c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:56.124659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66b14c20ff2427ab25ec24096c4317c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:56.155371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66b14c20ff2427ab25ec24096c4317c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:56.156717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-03e57fdc-3a16-4357-a61f-cf42ef0e3959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66b14c20ff2427ab25ec24096c4317c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:56.239938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 891/1784] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:55 2022] DELETE /identity/v3/users/a66b14c20ff2427ab25ec24096c4317c => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:56.633186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:56.635979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:56.676102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7463a3-26a0-407b-aaa8-ffe41c6855da admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:56.847541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:56.855569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:56.856255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:56.858239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] PATH_INFO: `/v3/projects/4a50129ee95f42dda024fecff91547c1` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:56.867267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a50129ee95f42dda024fecff91547c1)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:56.879067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:57.059455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a50129ee95f42dda024fecff91547c1', 'request_id': 'req-72f95749-f5bb-4225-96bf-9220b964f48b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:57.072054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a50129ee95f42dda024fecff91547c1', 'request_id': 'req-72f95749-f5bb-4225-96bf-9220b964f48b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:57.103246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a50129ee95f42dda024fecff91547c1', 'request_id': 'req-72f95749-f5bb-4225-96bf-9220b964f48b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:57.119511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] The token cache is being invalidate because project 4a50129ee95f42dda024fecff91547c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:57.119778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:57.120563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-72f95749-f5bb-4225-96bf-9220b964f48b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:57.122980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 894/1785] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:56 2022] DELETE /identity/v3/projects/4a50129ee95f42dda024fecff91547c1 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:57.128229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a995-3da7-43d3-9212-09c69dff3555 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:57.128807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a995-3da7-43d3-9212-09c69dff3555 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:57.128807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0a995-3da7-43d3-9212-09c69dff3555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:57.155215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b8c0a995-3da7-43d3-9212-09c69dff3555 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:57.157038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b8c0a995-3da7-43d3-9212-09c69dff3555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:57.323610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 892/1786] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:57.324478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c316-5d97-4a2a-b344-822e25f3980f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:57.324654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c316-5d97-4a2a-b344-822e25f3980f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:57.325183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894c316-5d97-4a2a-b344-822e25f3980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:57.397427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-2894c316-5d97-4a2a-b344-822e25f3980f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:57.399855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2894c316-5d97-4a2a-b344-822e25f3980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:57.428048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a995-3da7-43d3-9212-09c69dff3555 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:57.429284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 895/1787] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:57.452446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0a995-3da7-43d3-9212-09c69dff3555 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:57.466684 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:57.467873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:57.468393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:57.468772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:57.469676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:57.470085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:57.472163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:57.514204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 893/1788] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:57.514938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c316-5d97-4a2a-b344-822e25f3980f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:57.530188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894c316-5d97-4a2a-b344-822e25f3980f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:57.540487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:57.542546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:57.542868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:57.543160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:57.543831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:57.551365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:57.655040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 896/1789] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:57 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:50:57.662295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:57.676877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a674ee-aa6b-4c60-857d-da0383bd3ba7 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:57.805525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:57.806150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:57.806472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:57.806748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:57.825534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:57.831718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:57.853432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:57.890643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:57.891171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 894/1790] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:57 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:50:57.915365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e64d46-363e-4bd1-ab1e-bbd12348f90a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:57.922445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:57.923175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:57.923397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:57.923670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:57.924714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:57.926685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:57.966295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:57.988322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 897/1791] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:58.004048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2cc4d-bc9b-4e5e-b9a4-e6447af3a079 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:58.019135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:58.037581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.048282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.048282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.048282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:58.051501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:58.115814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 895/1792] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:58.122174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:58.156118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdb5c3-47f9-4ca2-910b-8457d1d0e176 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:58.178779 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1eb26-9790-4044-b097-431d65dc84ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:58.180002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:58.182473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:58.182675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1eb26-9790-4044-b097-431d65dc84ec admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.183176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1eb26-9790-4044-b097-431d65dc84ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.183599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1eb26-9790-4044-b097-431d65dc84ec admin admin] PATH_INFO: `/v3/projects/6d2d1bf06d1748e4b121dac9aafc487e/users/53031468eba54a98ab844b0207c89ec4/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.206462 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ba953-4f78-44d8-9ff1-a6bc577d552d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:58.226709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:58.227600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.227883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.230693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] PATH_INFO: `/v3/projects/c7d8d3258e31434d918dbe6f87286269` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.230693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d8d3258e31434d918dbe6f87286269)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:58.232204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:58.270512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1eb26-9790-4044-b097-431d65dc84ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2d1bf06d1748e4b121dac9aafc487e, user_id=53031468eba54a98ab844b0207c89ec4, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:58.277947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1eb26-9790-4044-b097-431d65dc84ec admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:58.339486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 898/1793] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:58 2022] PUT /identity/v3/projects/6d2d1bf06d1748e4b121dac9aafc487e/users/53031468eba54a98ab844b0207c89ec4/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:58.349936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66179b-c865-4c44-af49-90d0574b89b2 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.350437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66179b-c865-4c44-af49-90d0574b89b2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.350789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66179b-c865-4c44-af49-90d0574b89b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.371534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:58.379571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d8d3258e31434d918dbe6f87286269', 'request_id': 'req-7ee14459-50cd-431b-898c-cf3b37e3da58'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.406772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d8d3258e31434d918dbe6f87286269', 'request_id': 'req-7ee14459-50cd-431b-898c-cf3b37e3da58'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.414635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d8d3258e31434d918dbe6f87286269', 'request_id': 'req-7ee14459-50cd-431b-898c-cf3b37e3da58'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.420842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8e66179b-c865-4c44-af49-90d0574b89b2 None None] MFA Rules not processed for user `53031468eba54a98ab844b0207c89ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:58.422443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8e66179b-c865-4c44-af49-90d0574b89b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:58.439893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] The token cache is being invalidate because project c7d8d3258e31434d918dbe6f87286269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:50:58.440675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.446397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.452655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 896/1794] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:58 2022] DELETE /identity/v3/projects/c7d8d3258e31434d918dbe6f87286269 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:58.453738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:58.459740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:58.567545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee14459-50cd-431b-898c-cf3b37e3da58 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:58.576559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:58.577566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.577850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.578131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] PATH_INFO: `/v3/users/d0354711887c4ce3bd96d65692702fbb` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.579488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0354711887c4ce3bd96d65692702fbb)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:58.581787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:58.636459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 899/1795] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:50:58.637995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.638305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.638598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.678514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:58.679535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:58.683327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0354711887c4ce3bd96d65692702fbb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.684037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0354711887c4ce3bd96d65692702fbb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.703332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 900/1796] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:58.704186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:58.705630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:58.712256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=4,keystone:SELECT=80,keystone:DELETE=16 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:58.713280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0354711887c4ce3bd96d65692702fbb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.728493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daf52f-89d8-4e8a-acf1-14c9d5abf1b9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:58.734337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0354711887c4ce3bd96d65692702fbb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.735712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e5d1dcc-5877-40d4-99ae-58aab8e9411e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0354711887c4ce3bd96d65692702fbb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:50:58.752067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3174c-5c1f-469c-9d8e-689c1a51dbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:58.752806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3174c-5c1f-469c-9d8e-689c1a51dbb3 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.753869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3174c-5c1f-469c-9d8e-689c1a51dbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.755127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3174c-5c1f-469c-9d8e-689c1a51dbb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.756188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3174c-5c1f-469c-9d8e-689c1a51dbb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:58.761529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 897/1797] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:50:58 2022] DELETE /identity/v3/users/d0354711887c4ce3bd96d65692702fbb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:58.766305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14de9b3-7ac2-4c9d-963f-ddfdfc7c2465 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.766595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14de9b3-7ac2-4c9d-963f-ddfdfc7c2465 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.766885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3174c-5c1f-469c-9d8e-689c1a51dbb3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:58.768199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14de9b3-7ac2-4c9d-963f-ddfdfc7c2465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.831054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d14de9b3-7ac2-4c9d-963f-ddfdfc7c2465 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:58.834117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d14de9b3-7ac2-4c9d-963f-ddfdfc7c2465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:58.842077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 901/1798] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:58 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:58.842077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f76f-b3fc-401e-b07b-fbad3345d91c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.842077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f76f-b3fc-401e-b07b-fbad3345d91c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.842077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33f76f-b3fc-401e-b07b-fbad3345d91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.867852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8b33f76f-b3fc-401e-b07b-fbad3345d91c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:58.869150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8b33f76f-b3fc-401e-b07b-fbad3345d91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:58.961658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 898/1799] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:58.961658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14de9b3-7ac2-4c9d-963f-ddfdfc7c2465 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:58.978387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14de9b3-7ac2-4c9d-963f-ddfdfc7c2465 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:58.985444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 902/1800] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:50:58.985444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33f76f-b3fc-401e-b07b-fbad3345d91c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:58.989327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:58.990141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:58.990500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:58.991212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:58.992022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:58.994982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.001215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33f76f-b3fc-401e-b07b-fbad3345d91c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.006494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.007168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.008169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.008453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.009770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:50:59.010081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.012581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.029550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 903/1801] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:50:59.029764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 899/1802] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:58 2022] POST /identity/v3/users => generated 300 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:59.029860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.031400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.031575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:59.044636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c084aa3-147b-41a2-b04d-ea7ccdb3212e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.050408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.051335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.052060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.052588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.053639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.055495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.073768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 900/1803] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:59.075438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.090804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229a7af-5ab8-4626-8932-de83023def81 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.101801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.102985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.103419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.103781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.104619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.108345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.122860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b249fac-386d-49ff-8517-a277aa0ab1a2 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.126718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.127381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.127657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.127930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.134914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:59.151509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.155514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.169242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 901/1804] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:50:59 2022] POST /identity/v3/projects => generated 401 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:59.170174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.174468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 904/1805] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:50:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:50:59.176016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.189031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b77ea4-7a69-4a33-bfb7-770305d116c7 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.303494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.304984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.305482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.307102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.310365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.312744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.323755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1ef10-3793-44ea-a47a-8305849c2a3c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.331165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.332167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.332468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.333009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] PATH_INFO: `/v3/projects/f660cd46cdf848f3a936a7850ec7afbf/users/99fb2f5f80a042dca66709f88867b181/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.346799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 905/1806] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:50:59 2022] POST /identity/v3/users => generated 301 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:50:59.351873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.354364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f660cd46cdf848f3a936a7850ec7afbf, user_id=99fb2f5f80a042dca66709f88867b181, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.357246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.365522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15127bed-f4d0-4bb6-879f-6686cdaf6f61 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.371329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.372359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.372648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.373101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.373962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.375646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.393118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 906/1807] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:59.393316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 902/1808] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:59 2022] PUT /identity/v3/projects/f660cd46cdf848f3a936a7850ec7afbf/users/99fb2f5f80a042dca66709f88867b181/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:59.394667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.396371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.413130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cf677-99f5-4fa9-bb19-6dfc14dffe80 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.427369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:50:59.569130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679b23e-995e-48aa-91e6-69fe081c1e51 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.584991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.585613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.585897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.586170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.588449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.617628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.635640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 903/1809] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:59.653010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.678104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bed2f-64aa-4bce-bc25-9f44eee2e657 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:50:59.685023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e4cd5-c158-469d-b5d7-313b180066ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.686606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4cd5-c158-469d-b5d7-313b180066ab admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.689394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4cd5-c158-469d-b5d7-313b180066ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.690054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e4cd5-c158-469d-b5d7-313b180066ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.691192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e4cd5-c158-469d-b5d7-313b180066ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.692519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:50:59.700710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.701339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.701339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] PATH_INFO: `/v3/projects/f660cd46cdf848f3a936a7850ec7afbf/users/99fb2f5f80a042dca66709f88867b181/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.706742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e4cd5-c158-469d-b5d7-313b180066ab admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.721174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f660cd46cdf848f3a936a7850ec7afbf, user_id=99fb2f5f80a042dca66709f88867b181, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:50:59.724957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:50:59.748495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 904/1810] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:50:59 2022] PUT /identity/v3/projects/f660cd46cdf848f3a936a7850ec7afbf/users/99fb2f5f80a042dca66709f88867b181/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:50:59.749466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:50:59.753003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:50:59.768561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 907/1811] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:50:59.786356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1f12c-03c7-4264-bf30-fa94e37b9417 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:50:59.786665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1f12c-03c7-4264-bf30-fa94e37b9417 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:50:59.791663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1f12c-03c7-4264-bf30-fa94e37b9417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:50:59.928486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d0f1f12c-03c7-4264-bf30-fa94e37b9417 None None] MFA Rules not processed for user `99fb2f5f80a042dca66709f88867b181`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:50:59.931570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d0f1f12c-03c7-4264-bf30-fa94e37b9417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:00.063078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99276a-11b3-49e8-8665-da57e244e9c1 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:00.078816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:00.080348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:00.083978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:00.085489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] PATH_INFO: `/v3/projects/7d6934f696f84c2d88ed56193ddecada` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:00.088907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d6934f696f84c2d88ed56193ddecada)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:00.095121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:00.174543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:00.202618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 908/1812] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:50:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:00.203407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1f12c-03c7-4264-bf30-fa94e37b9417 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:00.235461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d6934f696f84c2d88ed56193ddecada', 'request_id': 'req-d67331d8-4979-4aab-8211-67e9abaae601'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:00.248556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1f12c-03c7-4264-bf30-fa94e37b9417 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:00.252165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d6934f696f84c2d88ed56193ddecada', 'request_id': 'req-d67331d8-4979-4aab-8211-67e9abaae601'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:00.263978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d6934f696f84c2d88ed56193ddecada', 'request_id': 'req-d67331d8-4979-4aab-8211-67e9abaae601'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:00.288757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] The token cache is being invalidate because project 7d6934f696f84c2d88ed56193ddecada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:00.289054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:00.289725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:00.291856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 905/1813] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:50:59 2022] DELETE /identity/v3/projects/7d6934f696f84c2d88ed56193ddecada => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:00.292840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:00.295573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:00.382524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:00.406292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8822ff3-7638-4b3c-a195-9f9e3971bb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:00.412501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822ff3-7638-4b3c-a195-9f9e3971bb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:00.412683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822ff3-7638-4b3c-a195-9f9e3971bb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:00.413490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8822ff3-7638-4b3c-a195-9f9e3971bb94 admin admin] PATH_INFO: `/v3/projects/ee60af06322b445bae724941f887db68/users/32ad06046e7b4f5098c62844a6a86d98/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:00.422899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67331d8-4979-4aab-8211-67e9abaae601 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:00.426924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:00.427661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:00.427939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:00.428252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:00.430110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:00.471154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822ff3-7638-4b3c-a195-9f9e3971bb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee60af06322b445bae724941f887db68, user_id=32ad06046e7b4f5098c62844a6a86d98, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:00.482182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8822ff3-7638-4b3c-a195-9f9e3971bb94 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:00.512110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 909/1814] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:00 2022] PUT /identity/v3/projects/ee60af06322b445bae724941f887db68/users/32ad06046e7b4f5098c62844a6a86d98/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:00.515714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:00.516973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:00.517396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:00.554779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:00.555053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] MFA Rules not processed for user `5e17d25a83fe48d99ddbd209fbea8486`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:00.556458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:00.557193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:00.587207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 906/1815] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:00.592955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:00.595429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:00.638225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6101de-0d5a-4d79-8ff9-4f95f6ff3f3f service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:00.718213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 910/1816] 10.222.0.71 () {58 vars in 1015 bytes} [Thu Jun 30 15:51:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:00.720661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:00.728718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:00.748216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefea82-3a42-456a-afda-b81a1c157306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:00.748845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefea82-3a42-456a-afda-b81a1c157306 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:00.749213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefea82-3a42-456a-afda-b81a1c157306 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:00.749444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefea82-3a42-456a-afda-b81a1c157306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:00.751573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6aefea82-3a42-456a-afda-b81a1c157306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:00.945770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefea82-3a42-456a-afda-b81a1c157306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:00.951982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefea82-3a42-456a-afda-b81a1c157306 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:00.960594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12fd7a-ad65-4018-974b-a37d26a5707f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:00.986729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:00.987818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:00.988138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:00.988462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] PATH_INFO: `/v3/users/c1ee9c16aadd4ac5a753f44fd0a70483` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:00.995115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ee9c16aadd4ac5a753f44fd0a70483)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:01.005234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:01.022473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 907/1817] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:01.036346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:01.036719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:01.037003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:01.103522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:01.175438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ee9c16aadd4ac5a753f44fd0a70483'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.176088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] MFA Rules not processed for user `32ad06046e7b4f5098c62844a6a86d98`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:01.177523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.214150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ee9c16aadd4ac5a753f44fd0a70483'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.232935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:01.238843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ee9c16aadd4ac5a753f44fd0a70483'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.240219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ee9c16aadd4ac5a753f44fd0a70483'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.243628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ee9c16aadd4ac5a753f44fd0a70483'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.302054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 911/1818] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:00 2022] DELETE /identity/v3/users/c1ee9c16aadd4ac5a753f44fd0a70483 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:01.321959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:01.325138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.459529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 908/1819] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:01.459529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:01.460266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.491770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b06fa4-8a02-4da7-8ff8-da0e6c35648a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:01.497277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:01.498015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5e754-b020-4c8c-8c63-cab006720a7f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:01.498875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:01.499446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:01.500166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:01.503405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.512698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:01.514527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:01.514896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:01.515407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:01.521421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:01.525478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:01.530366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:01.532393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:01.542865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 912/1820] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:01.544419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:01.547923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 909/1821] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:01.549432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:01.552067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.573461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaf7b0-3e5a-4980-9d5c-eae07ee4441d service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:01.589367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:01.590341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:01.590845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:01.591480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:01.613053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:01.615049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:01.638946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 913/1822] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:01.638946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:01.660700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.734217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19970747-a164-44b9-b5f5-de1e13daa9b3 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:01.741457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:01.742702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:01.743293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:01.743737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] PATH_INFO: `/v3/users/53031468eba54a98ab844b0207c89ec4` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:01.746404 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53031468eba54a98ab844b0207c89ec4)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:01.749148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:01.842343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227b162-c63a-4a6e-9921-67dddd2139a1 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:01.852358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:01.853736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:01.854264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53031468eba54a98ab844b0207c89ec4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.854516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:01.855334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:01.855881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53031468eba54a98ab844b0207c89ec4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.856790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:01.861335 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:01.889161 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53031468eba54a98ab844b0207c89ec4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.910415 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53031468eba54a98ab844b0207c89ec4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.911060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e83d115b-97b3-44e2-a367-6f7657f059b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53031468eba54a98ab844b0207c89ec4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:01.925612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 914/1823] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:01 2022] POST /identity/v3/projects => generated 401 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:01.930979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:01.938161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.946932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 910/1824] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:01 2022] DELETE /identity/v3/users/53031468eba54a98ab844b0207c89ec4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:01.946932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a0bb2-def2-4168-ae6a-a9145dede892 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:01.946932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a0bb2-def2-4168-ae6a-a9145dede892 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:01.946932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a0bb2-def2-4168-ae6a-a9145dede892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:01.986692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-600a0bb2-def2-4168-ae6a-a9145dede892 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:01.987936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-600a0bb2-def2-4168-ae6a-a9145dede892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:01.998553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:02.065236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:02.119509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e400201-85f6-400b-80ab-72e3741e770b admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:02.186245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 911/1825] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:02.246435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:02.246787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:02.247259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:02.247542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:02.250702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:02.256113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:02.306382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 915/1826] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:01 2022] POST /identity/v3/users => generated 307 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:02.308173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a0bb2-def2-4168-ae6a-a9145dede892 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:02.325969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:02.329119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:02.333238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a0bb2-def2-4168-ae6a-a9145dede892 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:02.351003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:02.354130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:02.356349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:02.356635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:02.360860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:02.366131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:02.411397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 912/1827] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:02.412562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:02.434306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973355a4-1533-4a66-ab20-6809881edb6c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:02.446388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:02.447238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:02.447609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:02.447888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] PATH_INFO: `/v3/projects/e544780976ae4d30a5e2b81f627f4af0/users/a02baae731304db3b09aa7404e667fe2/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:02.467612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9907a-1a53-4027-94f0-8547ebbd4c45 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:02.481830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:02.482707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:02.482994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:02.483317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:02.486166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e544780976ae4d30a5e2b81f627f4af0, user_id=a02baae731304db3b09aa7404e667fe2, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:02.492332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:02.515862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:02.518848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:02.521115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 913/1828] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:02 2022] PUT /identity/v3/projects/e544780976ae4d30a5e2b81f627f4af0/users/a02baae731304db3b09aa7404e667fe2/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:02.525039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:02.539216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 916/1829] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:02.688488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12227974-a983-42ae-9f86-e5f79c95af46 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:02.695504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829d72a-cabe-4789-b5b0-00f4be41e038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:02.705212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829d72a-cabe-4789-b5b0-00f4be41e038 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:02.705467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829d72a-cabe-4789-b5b0-00f4be41e038 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:02.706521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829d72a-cabe-4789-b5b0-00f4be41e038 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:02.706858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829d72a-cabe-4789-b5b0-00f4be41e038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:02.709836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829d72a-cabe-4789-b5b0-00f4be41e038 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:02.731475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 914/1830] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:02.735275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:02.752655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7013700-8fe1-4774-a477-e86b34885e75 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:02.763634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:02.765758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:02.766048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:02.766523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] PATH_INFO: `/v3/projects/e544780976ae4d30a5e2b81f627f4af0/users/a02baae731304db3b09aa7404e667fe2/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:02.774232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e544780976ae4d30a5e2b81f627f4af0, user_id=a02baae731304db3b09aa7404e667fe2, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:02.799493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:02.823013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 917/1831] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:02 2022] PUT /identity/v3/projects/e544780976ae4d30a5e2b81f627f4af0/users/a02baae731304db3b09aa7404e667fe2/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:02.834121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783942a9-a623-4058-bd64-80f519f469c5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:02.834558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783942a9-a623-4058-bd64-80f519f469c5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:02.834981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783942a9-a623-4058-bd64-80f519f469c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:02.906657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-783942a9-a623-4058-bd64-80f519f469c5 None None] MFA Rules not processed for user `a02baae731304db3b09aa7404e667fe2`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:02.910076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-783942a9-a623-4058-bd64-80f519f469c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:03.037918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 915/1832] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:02 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:03.061635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:03.063436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:03.211781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:03.241780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4065a7-1bad-4283-bde7-f55cbdf7f44f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:03.246053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:03.246732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:03.247020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:03.247296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:03.248129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ecd6c-b8d6-4817-8b33-2fa16456921b None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:03.248533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ecd6c-b8d6-4817-8b33-2fa16456921b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:03.249170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ecd6c-b8d6-4817-8b33-2fa16456921b None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:03.256925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 916/1833] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:51:03 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:51:03.267358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:03.270373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:03.272544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:03.277839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:03.287399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:03.310971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 918/1834] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:03.320803 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:03.331088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:03.339582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:03.410794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:03.426136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9b268-90b0-48f1-b2f7-5acdcfacccdd service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:03.434629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:03.435964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:03.436472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:03.436951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:03.441547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:03.517997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 917/1835] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:51:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:03.631370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:03.634212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:03.650900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 919/1836] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:03.746307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:03.749186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:03.810551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:03.812270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:03.843777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410164-473b-46b5-ae00-ba78b169eee3 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:03.865056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b7b2e-178d-4d08-84f0-fcf3f01b04fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:03.866542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b7b2e-178d-4d08-84f0-fcf3f01b04fa service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:03.868533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b7b2e-178d-4d08-84f0-fcf3f01b04fa service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:03.870778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b7b2e-178d-4d08-84f0-fcf3f01b04fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:03.923088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b7b2e-178d-4d08-84f0-fcf3f01b04fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:03.927583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1b7b2e-178d-4d08-84f0-fcf3f01b04fa service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:03.953766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 920/1837] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:03.979124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ab820-d1a0-439a-b4ae-6c1bbecd8bd2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:03.984173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:03.985113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:03.985399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:03.985678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] PATH_INFO: `/v3/projects/bc9753f32875428a87c4cef8007e4235` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:03.987013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc9753f32875428a87c4cef8007e4235)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:03.998657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:04.212855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc9753f32875428a87c4cef8007e4235', 'request_id': 'req-3f97a000-3911-4557-a69e-971eaee744b1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:04.233280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc9753f32875428a87c4cef8007e4235', 'request_id': 'req-3f97a000-3911-4557-a69e-971eaee744b1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:04.244201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc9753f32875428a87c4cef8007e4235', 'request_id': 'req-3f97a000-3911-4557-a69e-971eaee744b1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:04.260733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] The token cache is being invalidate because project bc9753f32875428a87c4cef8007e4235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:04.261159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:04.262061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3f97a000-3911-4557-a69e-971eaee744b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:04.272272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 918/1838] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:03 2022] DELETE /identity/v3/projects/bc9753f32875428a87c4cef8007e4235 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:04.278025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68803bcb-ada3-4840-80a3-fc99c92f2e0c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:04.292580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68803bcb-ada3-4840-80a3-fc99c92f2e0c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:04.293099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68803bcb-ada3-4840-80a3-fc99c92f2e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:04.332578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-68803bcb-ada3-4840-80a3-fc99c92f2e0c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:04.336211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-68803bcb-ada3-4840-80a3-fc99c92f2e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:04.518927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 921/1839] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:04.529573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da534-8502-4c05-8ca0-da41468e995a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:04.530386 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da534-8502-4c05-8ca0-da41468e995a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:04.531272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da534-8502-4c05-8ca0-da41468e995a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:04.588667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-710da534-8502-4c05-8ca0-da41468e995a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:04.591118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-710da534-8502-4c05-8ca0-da41468e995a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:04.605157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 919/1840] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:04.607055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68803bcb-ada3-4840-80a3-fc99c92f2e0c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:04.624345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68803bcb-ada3-4840-80a3-fc99c92f2e0c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:04.637705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:04.638473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:04.638957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:04.639312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:04.640606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:04.641922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:04.644514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:04.684773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710da534-8502-4c05-8ca0-da41468e995a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:04.686367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 922/1841] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:04.698451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710da534-8502-4c05-8ca0-da41468e995a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:04.718308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:04.719365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:04.719958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:04.720405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:04.721243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:04.722257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:04.722895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:04.727166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:04.761059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854842e7-c3d6-4538-b459-ddcbea615a67 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:04.783805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:04.785625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:04.787026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:04.787302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] PATH_INFO: `/v3/projects/6d2d1bf06d1748e4b121dac9aafc487e` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:04.790541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2d1bf06d1748e4b121dac9aafc487e)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:04.796214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:04.833449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 920/1842] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:04 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:04.839435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:04.854183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:04.961565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2d1bf06d1748e4b121dac9aafc487e', 'request_id': 'req-e1736d4f-8e31-4c3f-b365-a8b0638abc64'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:04.980819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2d1bf06d1748e4b121dac9aafc487e', 'request_id': 'req-e1736d4f-8e31-4c3f-b365-a8b0638abc64'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:04.985132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:04.998190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2d1bf06d1748e4b121dac9aafc487e', 'request_id': 'req-e1736d4f-8e31-4c3f-b365-a8b0638abc64'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.021423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] The token cache is being invalidate because project 6d2d1bf06d1748e4b121dac9aafc487e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:05.021713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.022592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.025528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 923/1843] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:04 2022] DELETE /identity/v3/projects/6d2d1bf06d1748e4b121dac9aafc487e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:05.031550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:05.034582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:05.045300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce33b99-cc9b-4d2e-be28-5c3b04043e2d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:05.159143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1736d4f-8e31-4c3f-b365-a8b0638abc64 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:05.173081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:05.174265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.175531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.175917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] PATH_INFO: `/v3/users/99fb2f5f80a042dca66709f88867b181` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.178046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99fb2f5f80a042dca66709f88867b181)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:05.184487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:05.215144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:05.217616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.226865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.227248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.229142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:05.233537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:05.289656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 921/1844] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:04 2022] POST /identity/v3/users => generated 285 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:05.302057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:05.315230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:05.331611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203257a-ebee-48af-9b2a-c948084cacd9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:05.337139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:05.338221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.338498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.338771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.341693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:05.343232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:05.358967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 922/1845] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:05.362469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:05.368435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99fb2f5f80a042dca66709f88867b181'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.380430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88040269-e3d9-4822-bdc8-ff15ca2729a3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:05.381869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99fb2f5f80a042dca66709f88867b181'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.387383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:05.388769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.389857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.390308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.391201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:05.393163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:05.394350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99fb2f5f80a042dca66709f88867b181'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.395737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99fb2f5f80a042dca66709f88867b181'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.397312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-13c670af-f074-45d0-9a5a-c1b2f060d714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99fb2f5f80a042dca66709f88867b181'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:05.452276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 923/1846] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:05.453257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:05.458025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:05.476038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 924/1847] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:05 2022] DELETE /identity/v3/users/99fb2f5f80a042dca66709f88867b181 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:05.492704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f8a85-83fa-4975-9538-78e69d2aacc2 None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.493536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f8a85-83fa-4975-9538-78e69d2aacc2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.494088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f8a85-83fa-4975-9538-78e69d2aacc2 None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.497681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 925/1848] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:51:05 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:51:05.524027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e01b36-1c3b-44f5-b91e-651d2bf89759 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:05.549008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b09a92-a98e-412e-bfe9-91b26cc1552d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.549008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b09a92-a98e-412e-bfe9-91b26cc1552d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.549008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b09a92-a98e-412e-bfe9-91b26cc1552d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.590451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d7b09a92-a98e-412e-bfe9-91b26cc1552d None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:05.591890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d7b09a92-a98e-412e-bfe9-91b26cc1552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:05.707212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:05.736016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:05.737193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 926/1849] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:51:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:05.737327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.738009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.738295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] PATH_INFO: `/v3/projects/bdc3cd953f794a96b237dfbdf20a1240/users/0f80360f27ea4e8d9bf22b1a422289fa/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.757061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc3cd953f794a96b237dfbdf20a1240, user_id=0f80360f27ea4e8d9bf22b1a422289fa, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:05.760221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:05.777981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:05.788568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 924/1850] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:05 2022] PUT /identity/v3/projects/bdc3cd953f794a96b237dfbdf20a1240/users/0f80360f27ea4e8d9bf22b1a422289fa/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:05.798755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620245b9-4106-412f-b55e-f3286cb3ed14 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:05.799568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620245b9-4106-412f-b55e-f3286cb3ed14 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:05.800121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620245b9-4106-412f-b55e-f3286cb3ed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:05.870894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-620245b9-4106-412f-b55e-f3286cb3ed14 None None] MFA Rules not processed for user `0f80360f27ea4e8d9bf22b1a422289fa`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:05.873502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-620245b9-4106-412f-b55e-f3286cb3ed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:05.972627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:05.974576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:05.998518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 927/1851] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:05.999844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620245b9-4106-412f-b55e-f3286cb3ed14 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:06.149145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4784c2-52dc-4d79-af2e-7fddc7267e8c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.167058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620245b9-4106-412f-b55e-f3286cb3ed14 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.173114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.173916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.174274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.174616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.175474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:06.177667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.177880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:06.181563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.182091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.182503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.197329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:06.198939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:06.244408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 928/1852] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:05 2022] POST /identity/v3/projects => generated 369 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:06.245440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:06.247115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 925/1853] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:06.250371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:06.252343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:06.318386 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202975c-b244-4fc9-9fc0-1dabf3120441 service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.426871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99994344-5faa-4f49-a46c-ed051a3cb316 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.436324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.437023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.439340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.439689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.440985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:06.442597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:06.454965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.455864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.456179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.456444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] PATH_INFO: `/v3/projects/f660cd46cdf848f3a936a7850ec7afbf` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.465054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f660cd46cdf848f3a936a7850ec7afbf)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:06.471115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:06.505460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:06.507436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 926/1854] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:06 2022] POST /identity/v3/users => generated 291 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:06.508184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:06.534398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7993-6a67-45b4-a286-b0591a3b904d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.549154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.550104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.550550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.550975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.551871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:06.553667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:06.573418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 927/1855] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:06.578598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:06.598104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deed610-3e2f-4d52-b66d-876f08bd1915 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.630417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f660cd46cdf848f3a936a7850ec7afbf', 'request_id': 'req-acdb745d-1121-4d49-8293-7ac6cf91a626'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:06.646040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f660cd46cdf848f3a936a7850ec7afbf', 'request_id': 'req-acdb745d-1121-4d49-8293-7ac6cf91a626'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:06.669329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f660cd46cdf848f3a936a7850ec7afbf', 'request_id': 'req-acdb745d-1121-4d49-8293-7ac6cf91a626'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:06.694857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.695768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.696078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.696357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] PATH_INFO: `/v3/projects/17d2fc348118443a9b5e77838466bae7/users/f9c34274484b4f97b48f40e434691ffb/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.696742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] The token cache is being invalidate because project f660cd46cdf848f3a936a7850ec7afbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:06.697253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:06.699344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acdb745d-1121-4d49-8293-7ac6cf91a626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:06.707217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:06.709716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 929/1856] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:06 2022] DELETE /identity/v3/projects/f660cd46cdf848f3a936a7850ec7afbf => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:06.714873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811923b7-5ce0-46ff-9a15-1aa5e5b898ce None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.715316 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811923b7-5ce0-46ff-9a15-1aa5e5b898ce None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.715664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811923b7-5ce0-46ff-9a15-1aa5e5b898ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.719646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d2fc348118443a9b5e77838466bae7, user_id=f9c34274484b4f97b48f40e434691ffb, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:06.723907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:06.745446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-811923b7-5ce0-46ff-9a15-1aa5e5b898ce None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:06.746857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-811923b7-5ce0-46ff-9a15-1aa5e5b898ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:06.755805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 928/1857] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:06 2022] PUT /identity/v3/projects/17d2fc348118443a9b5e77838466bae7/users/f9c34274484b4f97b48f40e434691ffb/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:06.761968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:06.764642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:06.883201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 930/1858] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:06.906320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaea93-172b-4cf6-822f-5d51ba0b3f56 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.906648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaea93-172b-4cf6-822f-5d51ba0b3f56 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.908173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdaea93-172b-4cf6-822f-5d51ba0b3f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.917300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200456ac-3cab-4c4e-91c9-639ea68c1794 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.930856 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.932850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.933159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.934179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:06.934985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:06.938877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:06.964032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 929/1859] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:06.965311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:06.983214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ffdaea93-172b-4cf6-822f-5d51ba0b3f56 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:06.985905 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ffdaea93-172b-4cf6-822f-5d51ba0b3f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:06.991416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1a0e2-455f-401a-b2e4-4c10a2ff9830 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:06.997645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4275084-e9ee-4e50-833d-98b9deddd1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:06.998701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4275084-e9ee-4e50-833d-98b9deddd1df admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:06.998985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4275084-e9ee-4e50-833d-98b9deddd1df admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:06.999375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4275084-e9ee-4e50-833d-98b9deddd1df admin admin] PATH_INFO: `/v3/projects/17d2fc348118443a9b5e77838466bae7/users/f9c34274484b4f97b48f40e434691ffb/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.002027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4275084-e9ee-4e50-833d-98b9deddd1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d2fc348118443a9b5e77838466bae7, user_id=f9c34274484b4f97b48f40e434691ffb, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.005586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4275084-e9ee-4e50-833d-98b9deddd1df admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.018283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 931/1860] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:07.022978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaea93-172b-4cf6-822f-5d51ba0b3f56 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.034757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdaea93-172b-4cf6-822f-5d51ba0b3f56 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.036093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 930/1861] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:06 2022] PUT /identity/v3/projects/17d2fc348118443a9b5e77838466bae7/users/f9c34274484b4f97b48f40e434691ffb/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:07.042170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.042994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.043272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.091946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] MFA Rules not processed for user `f9c34274484b4f97b48f40e434691ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:07.093219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:07.169771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.169984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.170519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.170519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.171264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:07.171609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.173559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.196593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 931/1862] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:07.224700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.226036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 932/1863] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:07.226954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:07.230100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.248098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ea1b9-0063-4304-a76a-4f420792e599 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.259622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.260496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.260930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.261351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.262442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.265180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.319773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 933/1864] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:07 2022] POST /identity/v3/projects => generated 387 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:07.323955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.343040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:07.377182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1c0f2-079b-4e53-80be-e2b0045da0f7 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.396537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:07.445412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf65f4-f694-4a3f-b7ac-bfc358c63595 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.460648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.461602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.461895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.462169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.463041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.465200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.514547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.514547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.514814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.515087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.516042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 934/1865] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:07 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:07.517094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.534064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7076cb6-d850-43bf-ac89-657363addf2b admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.551615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.552348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.553147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.553147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.553605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.555236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.579644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 935/1866] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:07.585467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.610953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242166c-5399-47eb-942f-585a752000d4 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.625132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.625865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.626146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.626425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.626956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.627173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.628461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.628884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.642709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 932/1867] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:07 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:07.652595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 936/1868] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:07.654895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.667377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f76b85-a9e8-4acd-9bd8-a057561335b8 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.673889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c5c5f-0c12-47fc-98d8-764a653819e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.674925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5c5f-0c12-47fc-98d8-764a653819e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.675866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5c5f-0c12-47fc-98d8-764a653819e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.676334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c5c5f-0c12-47fc-98d8-764a653819e7 admin admin] PATH_INFO: `/v3/projects/da4cfcd0edd146d58625878f5b47444d/users/29b7df9d8e8f4570a125529327a0252a/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.699424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c5c5f-0c12-47fc-98d8-764a653819e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da4cfcd0edd146d58625878f5b47444d, user_id=29b7df9d8e8f4570a125529327a0252a, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.703459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c5c5f-0c12-47fc-98d8-764a653819e7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.716895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.724733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 933/1869] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:07 2022] PUT /identity/v3/projects/da4cfcd0edd146d58625878f5b47444d/users/29b7df9d8e8f4570a125529327a0252a/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:07.735889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb9403-8faa-427f-854c-dc2e91e5671a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.737307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb9403-8faa-427f-854c-dc2e91e5671a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.738443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb9403-8faa-427f-854c-dc2e91e5671a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.747838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbee2a8-2b08-41f1-9ebf-64c35d413b37 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:07.811507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1ccb9403-8faa-427f-854c-dc2e91e5671a None None] MFA Rules not processed for user `29b7df9d8e8f4570a125529327a0252a`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:07.812767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1ccb9403-8faa-427f-854c-dc2e91e5671a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:07.830709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:07.831682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:07.832373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:07.833173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:07.835804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:07.899334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 934/1870] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:07.903929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb9403-8faa-427f-854c-dc2e91e5671a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:07.921211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:07.923180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:07.935033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 937/1871] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:07 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:07.937905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:08.022158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb9403-8faa-427f-854c-dc2e91e5671a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:08.031723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:08.032453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.032743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.033607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.034512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:08.037695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:08.076757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 935/1872] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:07 2022] POST /identity/v3/projects => generated 389 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:08.084157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:08.117748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4c840-0993-4be0-b3b8-fcaefd31d356 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:08.203429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:08.286483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:08.287231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.287955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.288256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.289565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:08.293781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:08.330178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 938/1873] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:08 2022] POST /identity/v3/users => generated 301 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:08.333954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:08.348389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3730c488-d6d0-43a5-8bf7-dc2a5cce8329 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:08.359706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:08.360883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.361398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.361992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.368571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:08.376593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:08.394985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 936/1874] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:08.400433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:08.418663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41938c0e-f58e-4482-9369-7a917c6b38e5 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:08.427233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:08.428198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.428476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.428747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] PATH_INFO: `/v3/projects/0f91c080119746538228679fdac151de/users/c3c9c8af0e2b4f46b63b5e4c801d5f70/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.466633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f91c080119746538228679fdac151de, user_id=c3c9c8af0e2b4f46b63b5e4c801d5f70, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:08.471626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:08.501902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 939/1875] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:08 2022] PUT /identity/v3/projects/0f91c080119746538228679fdac151de/users/c3c9c8af0e2b4f46b63b5e4c801d5f70/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:08.503230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:08.525154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e68863-03c3-4596-a1fc-3ff342e2495a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:08.622831 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ec69a-99a6-4cf6-8beb-3222d0807b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:08.623724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ec69a-99a6-4cf6-8beb-3222d0807b39 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.624168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ec69a-99a6-4cf6-8beb-3222d0807b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.624535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ec69a-99a6-4cf6-8beb-3222d0807b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.625463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ec69a-99a6-4cf6-8beb-3222d0807b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:08.627267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ec69a-99a6-4cf6-8beb-3222d0807b39 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:08.640091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 937/1876] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:08.643213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:08.654443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887fb4a-cd7a-4671-b887-1464561adbd0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:08.659427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:08.660910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.660910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.660910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] PATH_INFO: `/v3/projects/0f91c080119746538228679fdac151de/users/c3c9c8af0e2b4f46b63b5e4c801d5f70/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.663467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f91c080119746538228679fdac151de, user_id=c3c9c8af0e2b4f46b63b5e4c801d5f70, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:08.669143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:08.687621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 940/1877] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:08 2022] PUT /identity/v3/projects/0f91c080119746538228679fdac151de/users/c3c9c8af0e2b4f46b63b5e4c801d5f70/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:08.697232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee615-a027-4097-8e3a-2e18001a57cb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.697453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee615-a027-4097-8e3a-2e18001a57cb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.697984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee615-a027-4097-8e3a-2e18001a57cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.723890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-87dee615-a027-4097-8e3a-2e18001a57cb None None] MFA Rules not processed for user `c3c9c8af0e2b4f46b63b5e4c801d5f70`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:08.725130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-87dee615-a027-4097-8e3a-2e18001a57cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:08.797757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 938/1878] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:08.810237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:08.872511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418c3f5-c15d-4e48-89eb-f61f9bb18f8d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:08.876425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c51e459-b870-46ab-bfd2-c6579f075037 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:08.877063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51e459-b870-46ab-bfd2-c6579f075037 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:08.877341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51e459-b870-46ab-bfd2-c6579f075037 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:08.877607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c51e459-b870-46ab-bfd2-c6579f075037 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:08.887042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51e459-b870-46ab-bfd2-c6579f075037 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:08.888715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c51e459-b870-46ab-bfd2-c6579f075037 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:08.898186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 941/1879] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:09.027461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dee615-a027-4097-8e3a-2e18001a57cb None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:09.054364 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:09.095094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dee615-a027-4097-8e3a-2e18001a57cb None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:09.099693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:09.100613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:09.100915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:09.101187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] PATH_INFO: `/v3/users/29b7df9d8e8f4570a125529327a0252a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:09.102388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b7df9d8e8f4570a125529327a0252a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:09.104458 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:09.163447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b7df9d8e8f4570a125529327a0252a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.164213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b7df9d8e8f4570a125529327a0252a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.178682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b7df9d8e8f4570a125529327a0252a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.189032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b7df9d8e8f4570a125529327a0252a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.189818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b7df9d8e8f4570a125529327a0252a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.203289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 939/1880] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:09 2022] DELETE /identity/v3/users/29b7df9d8e8f4570a125529327a0252a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:09.207898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:09.208227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:09.208506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:09.228516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:09.229788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:09.302305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 942/1881] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:09.312176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:09.313730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:09.372276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57914741-1680-4871-9070-dc27f087ef47 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:09.376630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:09.377275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:09.377547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:09.377802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:09.387888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:09.389420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:09.399245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 940/1882] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:09.778710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:09.780129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:09.793812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c8014-a11c-4fc2-964a-8cc5e66c7234 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:09.799556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:09.800461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:09.800739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:09.801009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] PATH_INFO: `/v3/projects/da4cfcd0edd146d58625878f5b47444d` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:09.802716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da4cfcd0edd146d58625878f5b47444d)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:09.804844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:09.834076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:09.865795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da4cfcd0edd146d58625878f5b47444d', 'request_id': 'req-642d6a44-7291-4043-869f-67432c16a308'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.874064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da4cfcd0edd146d58625878f5b47444d', 'request_id': 'req-642d6a44-7291-4043-869f-67432c16a308'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.883852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da4cfcd0edd146d58625878f5b47444d', 'request_id': 'req-642d6a44-7291-4043-869f-67432c16a308'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.899166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] The token cache is being invalidate because project da4cfcd0edd146d58625878f5b47444d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:09.899690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.900749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:09.905248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 943/1883] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:09 2022] DELETE /identity/v3/projects/da4cfcd0edd146d58625878f5b47444d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:09.910323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:09.926947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f437cb8-478e-4f97-9ef8-647edc7131f2 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:09.977407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:09.996097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:09.997040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:09.997443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:09.997742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] PATH_INFO: `/v3/users/c3c9c8af0e2b4f46b63b5e4c801d5f70` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:09.998947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c9c8af0e2b4f46b63b5e4c801d5f70)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:10.000915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:10.047092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c9c8af0e2b4f46b63b5e4c801d5f70'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.048145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c9c8af0e2b4f46b63b5e4c801d5f70'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.059633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c9c8af0e2b4f46b63b5e4c801d5f70'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.068164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c9c8af0e2b4f46b63b5e4c801d5f70'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.068794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8f319d60-c5a3-4762-8c62-28b23891a58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c9c8af0e2b4f46b63b5e4c801d5f70'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.080874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 941/1884] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:09 2022] DELETE /identity/v3/users/c3c9c8af0e2b4f46b63b5e4c801d5f70 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:10.405470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:10.406996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:10.422401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d6a44-7291-4043-869f-67432c16a308 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:10.480444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:10.481321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:10.481601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:10.481925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] PATH_INFO: `/v3/projects/0f91c080119746538228679fdac151de` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:10.483120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f91c080119746538228679fdac151de)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:10.485081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:10.537560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f91c080119746538228679fdac151de', 'request_id': 'req-df9b16e1-c733-4e1b-8681-b143a36c54ce'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.544111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f91c080119746538228679fdac151de', 'request_id': 'req-df9b16e1-c733-4e1b-8681-b143a36c54ce'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.553469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f91c080119746538228679fdac151de', 'request_id': 'req-df9b16e1-c733-4e1b-8681-b143a36c54ce'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.564331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] The token cache is being invalidate because project 0f91c080119746538228679fdac151de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:10.564595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.565252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-df9b16e1-c733-4e1b-8681-b143a36c54ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:10.566832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 944/1885] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:10 2022] DELETE /identity/v3/projects/0f91c080119746538228679fdac151de => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:10.571796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecde2f-4f28-41eb-8bf4-9cf3afed552a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:10.572093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecde2f-4f28-41eb-8bf4-9cf3afed552a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:10.572371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ecde2f-4f28-41eb-8bf4-9cf3afed552a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:10.590889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b0ecde2f-4f28-41eb-8bf4-9cf3afed552a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:10.592201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b0ecde2f-4f28-41eb-8bf4-9cf3afed552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:10.656870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 942/1886] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:10.664996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d3897-c1fe-4ab2-b036-42e53901aa45 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:10.665289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d3897-c1fe-4ab2-b036-42e53901aa45 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:10.665674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d3897-c1fe-4ab2-b036-42e53901aa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:10.688922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9e6d3897-c1fe-4ab2-b036-42e53901aa45 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:10.690205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9e6d3897-c1fe-4ab2-b036-42e53901aa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:10.703325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 945/1887] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:10.704459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecde2f-4f28-41eb-8bf4-9cf3afed552a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:10.716045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ecde2f-4f28-41eb-8bf4-9cf3afed552a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:10.724618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:10.725321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:10.725607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:10.725891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:10.726669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:10.726986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:10.728776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:10.743397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 943/1888] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:51:10.747594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d3897-c1fe-4ab2-b036-42e53901aa45 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:10.757463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d3897-c1fe-4ab2-b036-42e53901aa45 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:10.762975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:10.763678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:10.763955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:10.764259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:10.764984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:10.766954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:10.800279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 946/1889] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:10 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:10.801446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:10.812962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bb539-4978-4cef-bfe1-5fde506bd14c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:10.867220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:10.889219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:10.890189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:10.890700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:10.891053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:10.891848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:10.894081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.011889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 944/1890] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:10 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:11.019536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.039572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78e582-6796-4a40-bc75-3e614926340c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:11.045665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:11.046510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.046795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.047068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.047804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:11.058410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.058410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:11.078445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 947/1891] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:11.086014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.098022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa95b32-9e6f-44ba-8123-3c2d13447e54 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:11.110339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd3398-33dc-40f5-97cc-fddcb624cad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:11.110339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd3398-33dc-40f5-97cc-fddcb624cad1 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.110339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd3398-33dc-40f5-97cc-fddcb624cad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.110339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd3398-33dc-40f5-97cc-fddcb624cad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.110682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd3398-33dc-40f5-97cc-fddcb624cad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:11.113318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd3398-33dc-40f5-97cc-fddcb624cad1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.136990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 945/1892] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:11.137344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.150489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e51f2-9638-436b-bb46-c08e06004cd5 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:11.155695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:11.156939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.157318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.157709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] PATH_INFO: `/v3/projects/dcdc01f666a64a05a7e80dc0c8335b41/users/3fe5f3de75464458b40063b80856cb0f/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.184990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcdc01f666a64a05a7e80dc0c8335b41, user_id=3fe5f3de75464458b40063b80856cb0f, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:11.188436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.212557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 948/1893] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:11 2022] PUT /identity/v3/projects/dcdc01f666a64a05a7e80dc0c8335b41/users/3fe5f3de75464458b40063b80856cb0f/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:11.223077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5404a3-c186-4df7-b383-96c57c812a07 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.223857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5404a3-c186-4df7-b383-96c57c812a07 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.224213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5404a3-c186-4df7-b383-96c57c812a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.296413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8a5404a3-c186-4df7-b383-96c57c812a07 None None] MFA Rules not processed for user `3fe5f3de75464458b40063b80856cb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:11.299145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8a5404a3-c186-4df7-b383-96c57c812a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:11.427885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.428984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 946/1894] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:11.540663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a249633c-64d2-427e-ad7a-74ffc25cb366 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:11.554217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:11.555174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.555600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.556745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.557653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:11.559955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.597552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 949/1895] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:11 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:11.601804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5404a3-c186-4df7-b383-96c57c812a07 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.619915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5404a3-c186-4df7-b383-96c57c812a07 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:11.713625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:11.714574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.714860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.715690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.716490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:11.718967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.798790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 947/1896] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:11 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:11.803615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.819551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453472c-4f1d-4ac9-b1c1-cfddcbc7fb52 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:11.827495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:11.828498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.829804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.830410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.831348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:11.835412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.848838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 950/1897] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:11.852839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.873175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae490f-e60a-4b77-a352-dd7bcbc5bf91 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:11.879681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:11.880778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:11.881064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:11.881341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] PATH_INFO: `/v3/projects/fc766a7e580c431ca9fc3168d0bc62ab/users/0bd62767cde9495b9651e177b53d5f1d/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:11.906346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc766a7e580c431ca9fc3168d0bc62ab, user_id=0bd62767cde9495b9651e177b53d5f1d, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:11.910728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:11.945607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 948/1898] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:11 2022] PUT /identity/v3/projects/fc766a7e580c431ca9fc3168d0bc62ab/users/0bd62767cde9495b9651e177b53d5f1d/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:11.950568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.973083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:11.978646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:11.982313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094788a-c82a-4a34-8bad-c73297fc8a0e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:12.080239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:12.121858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:12.122779 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.124838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.124971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.126537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:12.132095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:12.158659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 951/1899] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:12.160065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:12.189095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83efe31-4e78-4d86-80e5-aecf0b441bd3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:12.199489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:12.206622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.207064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.207441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] PATH_INFO: `/v3/projects/fc766a7e580c431ca9fc3168d0bc62ab/users/0bd62767cde9495b9651e177b53d5f1d/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.210166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc766a7e580c431ca9fc3168d0bc62ab, user_id=0bd62767cde9495b9651e177b53d5f1d, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:12.213681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:12.221619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23b9da-bfc6-4f0d-8578-b282ace75431 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:12.253658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 952/1900] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:12 2022] PUT /identity/v3/projects/fc766a7e580c431ca9fc3168d0bc62ab/users/0bd62767cde9495b9651e177b53d5f1d/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:12.254801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:12.267883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:12.332334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:12.398050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e7a95-07df-4397-bf27-f83ac194bd13 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:12.404283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:12.405247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.406607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.406607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] PATH_INFO: `/v3/users/32ad06046e7b4f5098c62844a6a86d98` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.407979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ad06046e7b4f5098c62844a6a86d98)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:12.411429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:12.411878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:12.416683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.418236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.418670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] PATH_INFO: `/v3/users/0f80360f27ea4e8d9bf22b1a422289fa` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.423928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f80360f27ea4e8d9bf22b1a422289fa)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:12.426950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:12.517265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f80360f27ea4e8d9bf22b1a422289fa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.518214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f80360f27ea4e8d9bf22b1a422289fa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.533130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ad06046e7b4f5098c62844a6a86d98'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.538094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f80360f27ea4e8d9bf22b1a422289fa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.546210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ad06046e7b4f5098c62844a6a86d98'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.557589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ad06046e7b4f5098c62844a6a86d98'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.558399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ad06046e7b4f5098c62844a6a86d98'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.559074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbe9c40-6fb1-4d75-956c-99be5f7396a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ad06046e7b4f5098c62844a6a86d98'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.563151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f80360f27ea4e8d9bf22b1a422289fa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.564217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a8e06497-d4b2-4431-9a27-a4c20da4326f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f80360f27ea4e8d9bf22b1a422289fa'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:12.584027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 949/1901] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:11 2022] DELETE /identity/v3/users/0f80360f27ea4e8d9bf22b1a422289fa => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:12.586416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22108902-a534-4694-b5fa-fc527711b727 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.586939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22108902-a534-4694-b5fa-fc527711b727 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.587813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22108902-a534-4694-b5fa-fc527711b727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.600997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 953/1902] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:12 2022] DELETE /identity/v3/users/32ad06046e7b4f5098c62844a6a86d98 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:12.604218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c697b7-5b2b-44c0-b162-ec32bd89b9fb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.604218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c697b7-5b2b-44c0-b162-ec32bd89b9fb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.604218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c697b7-5b2b-44c0-b162-ec32bd89b9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.627682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-22108902-a534-4694-b5fa-fc527711b727 None None] MFA Rules not processed for user `0bd62767cde9495b9651e177b53d5f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:12.629360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-22108902-a534-4694-b5fa-fc527711b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:12.631738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-16c697b7-5b2b-44c0-b162-ec32bd89b9fb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:12.633397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-16c697b7-5b2b-44c0-b162-ec32bd89b9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:12.700360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:12.730810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 954/1903] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:12.732560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.732856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.733130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.756324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:12.757572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:12.763271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 950/1904] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:12.768762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 955/1905] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:12.777416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22108902-a534-4694-b5fa-fc527711b727 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:12.778802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-22108902-a534-4694-b5fa-fc527711b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:12.807165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:12.808845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:12.882729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22108902-a534-4694-b5fa-fc527711b727 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:12.889822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:12.891208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.892535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.894215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:12.895934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.913030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:12.914497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:12.917948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02877681-e882-49a8-9e7f-ae701655e89b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:12.926641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:12.927320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.927950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 951/1906] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:12 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:12.928972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:12.930350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:12.932166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.934423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.944217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e32194-5677-43b0-9e47-eff97ad0eaaf service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:12.952360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:12.952573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:12.953369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:12.953656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:12.954134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:12.955326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:12.965225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:12.966759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:12.976179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 956/1907] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:12.982337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 952/1908] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:13.027531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:13.039126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a974fb-8240-4b72-b87e-4bb33ed4fc35 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:13.043161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:13.044013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:13.044462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:13.044855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:13.046909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:13.128157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:13.129743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:13.140545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 957/1909] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:13.709622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:13.711090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:13.745486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff677d9-8a13-436a-88d5-731159af4b6c service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:13.759092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:13.760212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:13.760564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:13.760947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] PATH_INFO: `/v3/projects/bdc3cd953f794a96b237dfbdf20a1240` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:13.763795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc3cd953f794a96b237dfbdf20a1240)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:13.768045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:13.845016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc3cd953f794a96b237dfbdf20a1240', 'request_id': 'req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:13.854961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc3cd953f794a96b237dfbdf20a1240', 'request_id': 'req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:13.863844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc3cd953f794a96b237dfbdf20a1240', 'request_id': 'req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:13.877798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] The token cache is being invalidate because project bdc3cd953f794a96b237dfbdf20a1240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:13.878315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:13.878983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:13.880931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 953/1910] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:13 2022] DELETE /identity/v3/projects/bdc3cd953f794a96b237dfbdf20a1240 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:13.887088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:13.903148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d02822-8c45-4b65-8de0-ac0909d2fdfe service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:13.973988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:13.975231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:13.976215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:13.976829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] PATH_INFO: `/v3/users/f9c34274484b4f97b48f40e434691ffb` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:13.978210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c34274484b4f97b48f40e434691ffb)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:13.981478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:14.050523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c34274484b4f97b48f40e434691ffb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.062564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c34274484b4f97b48f40e434691ffb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.071543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c34274484b4f97b48f40e434691ffb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.072842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c34274484b4f97b48f40e434691ffb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.073626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c34274484b4f97b48f40e434691ffb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.096159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 958/1911] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:13 2022] DELETE /identity/v3/users/f9c34274484b4f97b48f40e434691ffb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:14.424672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:14.426569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:14.442741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2081a1-e8fb-42ad-aef6-ace3c4b9141a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:14.463374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:14.465004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:14.472108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:14.493215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:14.509576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:14.510507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:14.510786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:14.511054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] PATH_INFO: `/v3/projects/17d2fc348118443a9b5e77838466bae7` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:14.513924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d2fc348118443a9b5e77838466bae7)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:14.516078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:14.558830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1328c-419e-430d-b01b-c820454b359e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:14.564806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:14.566231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:14.566642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:14.566927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] PATH_INFO: `/v3/projects/ee60af06322b445bae724941f887db68` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:14.569329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee60af06322b445bae724941f887db68)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:14.571603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:14.588808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d2fc348118443a9b5e77838466bae7', 'request_id': 'req-4abef950-e491-46e6-a71b-8b8682e42b0c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.597189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d2fc348118443a9b5e77838466bae7', 'request_id': 'req-4abef950-e491-46e6-a71b-8b8682e42b0c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.605136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d2fc348118443a9b5e77838466bae7', 'request_id': 'req-4abef950-e491-46e6-a71b-8b8682e42b0c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.619238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] The token cache is being invalidate because project 17d2fc348118443a9b5e77838466bae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:14.619753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.620930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4abef950-e491-46e6-a71b-8b8682e42b0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.623506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 954/1912] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:14 2022] DELETE /identity/v3/projects/17d2fc348118443a9b5e77838466bae7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:14.628060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d13757-c574-4949-a8fd-aa3e74bfc690 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:14.628356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d13757-c574-4949-a8fd-aa3e74bfc690 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:14.628925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d13757-c574-4949-a8fd-aa3e74bfc690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:14.642119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee60af06322b445bae724941f887db68', 'request_id': 'req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.650006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee60af06322b445bae724941f887db68', 'request_id': 'req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.654170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-28d13757-c574-4949-a8fd-aa3e74bfc690 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:14.656678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-28d13757-c574-4949-a8fd-aa3e74bfc690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:14.662961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee60af06322b445bae724941f887db68', 'request_id': 'req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.690529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] The token cache is being invalidate because project ee60af06322b445bae724941f887db68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:14.690771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.691436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.696031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 959/1913] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:14 2022] DELETE /identity/v3/projects/ee60af06322b445bae724941f887db68 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:14.698541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:14.704601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:14.752317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 955/1914] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:14.762152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9c4af-fcef-440c-a47f-40d7fb3ae1a2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:14.762449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9c4af-fcef-440c-a47f-40d7fb3ae1a2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:14.762850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9c4af-fcef-440c-a47f-40d7fb3ae1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:14.783692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-26e9c4af-fcef-440c-a47f-40d7fb3ae1a2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:14.785238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-26e9c4af-fcef-440c-a47f-40d7fb3ae1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:14.798638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 956/1915] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:14.800650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9c4af-fcef-440c-a47f-40d7fb3ae1a2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:14.805833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd609c-95d4-4d6c-af32-cf41e4fbe8c9 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:14.811501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9c4af-fcef-440c-a47f-40d7fb3ae1a2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:14.816785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:14.817743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:14.818008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:14.818132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:14.818298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] PATH_INFO: `/v3/users/a02baae731304db3b09aa7404e667fe2` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:14.818621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:14.818895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:14.819196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:14.819971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:14.820115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02baae731304db3b09aa7404e667fe2)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:14.820623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:14.822171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:14.822445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:14.839444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 957/1916] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:14.843914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:14.869193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c159458-229c-4142-8597-0de7037eb116 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:14.875022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:14.875851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:14.876238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:14.876703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:14.877677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:14.878221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:14.881394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:14.897043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02baae731304db3b09aa7404e667fe2'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.909326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02baae731304db3b09aa7404e667fe2'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.926841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 958/1917] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:14 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:14.930637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02baae731304db3b09aa7404e667fe2'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.930956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:14.932998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02baae731304db3b09aa7404e667fe2'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.933661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02baae731304db3b09aa7404e667fe2'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:14.960682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 960/1918] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:14 2022] DELETE /identity/v3/users/a02baae731304db3b09aa7404e667fe2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:14.991146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:15.022195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8e635-3180-4522-b6de-fe47d427c22b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.100914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.101962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.102280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.102568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.103325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.105593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.136044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 959/1919] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:14 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:15.138832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.140447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:15.156648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ef458-b037-464c-8428-294b8dd4f9da admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.162692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.163306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.163601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.163922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.165386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.167062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.184062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.185074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 961/1920] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:15.221755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf62ef4-f95d-4333-8be0-57eb1c02014e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.235839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8947cc-784e-42d9-a8b8-12e8e388b6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.236562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8947cc-784e-42d9-a8b8-12e8e388b6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.236847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8947cc-784e-42d9-a8b8-12e8e388b6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.237124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8947cc-784e-42d9-a8b8-12e8e388b6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.237865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8947cc-784e-42d9-a8b8-12e8e388b6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.240610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8947cc-784e-42d9-a8b8-12e8e388b6f8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.258917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 960/1921] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:15.260035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.276403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a74072-4788-401d-bd5b-a087df251f79 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.284312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.287708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.288102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.288465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] PATH_INFO: `/v3/projects/ce2f5182f6554fa7a1c0c651c172ca8c/users/610d591a6fd24ed6980abc68416326fc/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.309936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2f5182f6554fa7a1c0c651c172ca8c, user_id=610d591a6fd24ed6980abc68416326fc, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.312713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.338737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 962/1922] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:15 2022] PUT /identity/v3/projects/ce2f5182f6554fa7a1c0c651c172ca8c/users/610d591a6fd24ed6980abc68416326fc/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:15.340253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.340546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.340826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.370970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] MFA Rules not processed for user `610d591a6fd24ed6980abc68416326fc`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:15.372769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:15.448105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 961/1923] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:15.452647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.529557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.531250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:15.532202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617c7c99-569c-42ce-b2ef-8881dddae602 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.539304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.540044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.540371 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.540722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.541850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.544614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.549765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa651-c5d2-434b-a3da-1819ff1409d4 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.555217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.556126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.556447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.556689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] PATH_INFO: `/v3/projects/e544780976ae4d30a5e2b81f627f4af0` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.557973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e544780976ae4d30a5e2b81f627f4af0)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.559870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.577014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 963/1924] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:15 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:15.581093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.614251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:15.615863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e544780976ae4d30a5e2b81f627f4af0', 'request_id': 'req-b368283b-ecef-4bae-b716-f43ccafdc497'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:15.625482 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e544780976ae4d30a5e2b81f627f4af0', 'request_id': 'req-b368283b-ecef-4bae-b716-f43ccafdc497'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:15.632900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e544780976ae4d30a5e2b81f627f4af0', 'request_id': 'req-b368283b-ecef-4bae-b716-f43ccafdc497'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:15.643308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] The token cache is being invalidate because project e544780976ae4d30a5e2b81f627f4af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:15.643588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:15.644251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-b368283b-ecef-4bae-b716-f43ccafdc497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:15.646069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 962/1925] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:15 2022] DELETE /identity/v3/projects/e544780976ae4d30a5e2b81f627f4af0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:15.648015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5b774-4e1d-49e4-b073-fe85b0a72b25 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.650710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d97f1-7b15-4ad3-9a45-d914202db1d3 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.650990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d97f1-7b15-4ad3-9a45-d914202db1d3 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.651257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d97f1-7b15-4ad3-9a45-d914202db1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.656091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:15.670154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f46d97f1-7b15-4ad3-9a45-d914202db1d3 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:15.671300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f46d97f1-7b15-4ad3-9a45-d914202db1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:15.707728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.708419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.708692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.708965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.709656 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.711608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.732620 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 964/1926] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:15 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:15.734122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 963/1927] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:15.741587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.743201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951c0f0-a981-48e3-bbf6-2dd06cca3d20 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.743364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:15.743533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951c0f0-a981-48e3-bbf6-2dd06cca3d20 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.743801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9951c0f0-a981-48e3-bbf6-2dd06cca3d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.756091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c076fb9-960a-41b3-820b-0447739ed9f8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.760641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.761327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.761604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.761789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9951c0f0-a981-48e3-bbf6-2dd06cca3d20 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:15.762264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.762591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.762992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9951c0f0-a981-48e3-bbf6-2dd06cca3d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:15.764261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.771069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 964/1928] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:15.775165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951c0f0-a981-48e3-bbf6-2dd06cca3d20 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.775286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 965/1929] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:15.779071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.784603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9951c0f0-a981-48e3-bbf6-2dd06cca3d20 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.788879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de4178-d971-42e9-9b00-c56b75c302d3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.789236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.789884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.790166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.790443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.791165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:15.791589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.793234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.793584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.794463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.794901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.795174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] PATH_INFO: `/v3/projects/155a1408a72e404f99a059ce6748d9e5/users/c9bdf4cf9c474a199592c3ad4c2d59b5/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.810816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 965/1930] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:51:15.813256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155a1408a72e404f99a059ce6748d9e5, user_id=c9bdf4cf9c474a199592c3ad4c2d59b5, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.815147 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.815886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.825440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0b99f-feeb-49ec-b994-0000aaac18ec admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.831626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 966/1931] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:15 2022] PUT /identity/v3/projects/155a1408a72e404f99a059ce6748d9e5/users/c9bdf4cf9c474a199592c3ad4c2d59b5/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:15.835533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.892024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.892702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.892985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.893260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.893998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.895913 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.897507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b96a5b5-944c-4944-945c-6f7df500adc7 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.902372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:15.903068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:15.903442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:15.903726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:15.904536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:15.906186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:15.918064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 967/1932] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:15.922116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.927384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 966/1933] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:15 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:15.931307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:15.950622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4540d5-b4a3-4b4e-a7e6-b2f07eb585fe admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:15.981053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:16.002419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e90c2-e00f-40e9-9cca-9f23042dbdba admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.008163 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.009790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.009915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.010120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.010837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.013134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.024581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e1606-80f2-4985-9a02-75c9bdae962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.025669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e1606-80f2-4985-9a02-75c9bdae962b admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.026008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e1606-80f2-4985-9a02-75c9bdae962b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.026239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e1606-80f2-4985-9a02-75c9bdae962b admin admin] PATH_INFO: `/v3/projects/155a1408a72e404f99a059ce6748d9e5/users/c9bdf4cf9c474a199592c3ad4c2d59b5/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.033912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e1606-80f2-4985-9a02-75c9bdae962b admin admin] RBAC: Authorizing `identity:create_grant(project_id=155a1408a72e404f99a059ce6748d9e5, user_id=c9bdf4cf9c474a199592c3ad4c2d59b5, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.035598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 967/1934] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:15 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:16.037409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e1606-80f2-4985-9a02-75c9bdae962b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.040481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.042901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:16.057873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 968/1935] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:15 2022] PUT /identity/v3/projects/155a1408a72e404f99a059ce6748d9e5/users/c9bdf4cf9c474a199592c3ad4c2d59b5/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:16.057989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbe62-2345-49c7-a41b-e4815b303985 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.070037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.070684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.071093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.114949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] MFA Rules not processed for user `c9bdf4cf9c474a199592c3ad4c2d59b5`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:16.120234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:16.163259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.164027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.164504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.164844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.165714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.167839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.182391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 968/1936] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:16.186298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.196699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b171961-9517-4349-af66-47d79ceef00f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.202048 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.202799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.203158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.203809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.204316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.206021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.221211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 969/1937] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:16.222401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.236774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49a32c-29a9-4cdc-8060-fef8488aa22a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.238750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 969/1938] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:16.245499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667977e-5c8c-4c7a-b077-a5d414511f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.250043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.250294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667977e-5c8c-4c7a-b077-a5d414511f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.251407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:16.251549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667977e-5c8c-4c7a-b077-a5d414511f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.252211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667977e-5c8c-4c7a-b077-a5d414511f04 admin admin] PATH_INFO: `/v3/projects/fff4a4452f6444eeb70ad4f0db32efda/users/6bd48b85826c4c0e94ab606acc9f28a3/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.276547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667977e-5c8c-4c7a-b077-a5d414511f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff4a4452f6444eeb70ad4f0db32efda, user_id=6bd48b85826c4c0e94ab606acc9f28a3, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.279799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667977e-5c8c-4c7a-b077-a5d414511f04 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.296902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 970/1939] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:16 2022] PUT /identity/v3/projects/fff4a4452f6444eeb70ad4f0db32efda/users/6bd48b85826c4c0e94ab606acc9f28a3/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:16.310966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8a1fe-36a5-4020-9ddb-5530ca76e82b None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.311173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8a1fe-36a5-4020-9ddb-5530ca76e82b None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.311507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8a1fe-36a5-4020-9ddb-5530ca76e82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.330425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec63dbe-dee3-42c5-84c0-77b65ee08623 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.342623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e6f8a1fe-36a5-4020-9ddb-5530ca76e82b None None] MFA Rules not processed for user `6bd48b85826c4c0e94ab606acc9f28a3`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:16.343816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e6f8a1fe-36a5-4020-9ddb-5530ca76e82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:16.383728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.384539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.384887 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.385222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.417436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 971/1940] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:16.421728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8a1fe-36a5-4020-9ddb-5530ca76e82b None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.441345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:16.449219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:16.456398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.457863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.469113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 970/1941] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:16.506204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8a1fe-36a5-4020-9ddb-5530ca76e82b None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.513136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.513826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.514107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.514368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.515097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.517301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.572483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 972/1942] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:16 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:16.575906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.593998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4c085-6020-4c40-9f36-fa2931746b4e service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.650220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.745483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.746300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.746584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.747210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.748407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.750522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.764731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510f105-cf69-4fc7-af94-18f33a5ee436 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.768812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.769664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.769953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.770233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.772232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:16.817423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 971/1943] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:16 2022] POST /identity/v3/users => generated 300 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:16.821375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.850136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53728e-8265-4a67-8c4a-513e27460ecd admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.856406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.857292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.857575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.858086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.858802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.860578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.876742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 972/1944] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:16.878107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.886241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.887966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.891784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac556960-d44d-4f70-8138-91cf5b3b9dac admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:16.899531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:16.901066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:16.901284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 973/1945] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:16.901634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:16.902252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] PATH_INFO: `/v3/projects/22ff62eefc054e00ae80f8312b664744/users/183d3a89db1347b597894074686a0798/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:16.921404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ff62eefc054e00ae80f8312b664744, user_id=183d3a89db1347b597894074686a0798, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:16.924192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:16.949353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 973/1946] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:16 2022] PUT /identity/v3/projects/22ff62eefc054e00ae80f8312b664744/users/183d3a89db1347b597894074686a0798/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:16.955767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:16.967864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de0343-dd11-4cd0-a9c7-4bb61bdc728d service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:17.072487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b822ffe-0488-46f6-b89e-1d92383de1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:17.073502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b822ffe-0488-46f6-b89e-1d92383de1ce admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:17.074046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b822ffe-0488-46f6-b89e-1d92383de1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:17.074643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b822ffe-0488-46f6-b89e-1d92383de1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:17.076692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b822ffe-0488-46f6-b89e-1d92383de1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:17.079087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b822ffe-0488-46f6-b89e-1d92383de1ce admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:17.082811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:17.099162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:17.112376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 974/1947] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:17.116234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902345a-407a-429c-b30d-6b8dde124083 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:17.123936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:17.124968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:17.125139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:17.125411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] PATH_INFO: `/v3/projects/22ff62eefc054e00ae80f8312b664744/users/183d3a89db1347b597894074686a0798/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:17.127372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ff62eefc054e00ae80f8312b664744, user_id=183d3a89db1347b597894074686a0798, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:17.131326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:17.144703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 974/1948] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:17 2022] PUT /identity/v3/projects/22ff62eefc054e00ae80f8312b664744/users/183d3a89db1347b597894074686a0798/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:17.164014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc30c7e-105d-4313-bfbd-3ae4fe211566 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:17.164284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc30c7e-105d-4313-bfbd-3ae4fe211566 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:17.164561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc30c7e-105d-4313-bfbd-3ae4fe211566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:17.215090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-adc30c7e-105d-4313-bfbd-3ae4fe211566 None None] MFA Rules not processed for user `183d3a89db1347b597894074686a0798`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:17.216968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-adc30c7e-105d-4313-bfbd-3ae4fe211566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:17.336865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 975/1949] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:17.358886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:17.360876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:17.495742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1163e8-4bdf-407f-91e0-fb4edf5b2ae0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:17.500149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161a464-eaf5-44ab-8970-84f388b30626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:17.501000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a464-eaf5-44ab-8970-84f388b30626 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:17.501542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a464-eaf5-44ab-8970-84f388b30626 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:17.502094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161a464-eaf5-44ab-8970-84f388b30626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:17.516314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a464-eaf5-44ab-8970-84f388b30626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:17.517773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161a464-eaf5-44ab-8970-84f388b30626 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:17.539754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 975/1950] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:18.227733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc30c7e-105d-4313-bfbd-3ae4fe211566 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:18.311425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc30c7e-105d-4313-bfbd-3ae4fe211566 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:18.317108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:18.318005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:18.318377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:18.318656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] PATH_INFO: `/v3/users/6bd48b85826c4c0e94ab606acc9f28a3` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:18.319918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd48b85826c4c0e94ab606acc9f28a3)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:18.321888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:18.394226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd48b85826c4c0e94ab606acc9f28a3'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:18.394923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd48b85826c4c0e94ab606acc9f28a3'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:18.412771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd48b85826c4c0e94ab606acc9f28a3'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:18.422991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd48b85826c4c0e94ab606acc9f28a3'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:18.424528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd48b85826c4c0e94ab606acc9f28a3'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:18.490119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:18.491286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:18.491822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:18.493331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 976/1951] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:18 2022] DELETE /identity/v3/users/6bd48b85826c4c0e94ab606acc9f28a3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:18.517101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:18.518343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:18.552419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:18.626463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 976/1952] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:18.646475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:18.648162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:18.655602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:18.771178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fced38e-f650-4361-bf9b-4870626123d9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:18.785057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f40c2c-6c85-47a6-9828-9233730504a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:18.785481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f40c2c-6c85-47a6-9828-9233730504a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:18.785777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f40c2c-6c85-47a6-9828-9233730504a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:18.788299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f40c2c-6c85-47a6-9828-9233730504a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:18.800167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f40c2c-6c85-47a6-9828-9233730504a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:18.801406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f40c2c-6c85-47a6-9828-9233730504a7 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:18.813345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 977/1953] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:18.895408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:18.897165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:18.911517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52064e7e-3c87-4001-8188-2c7df623d61a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:18.917018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:18.917922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:18.918482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:18.918482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] PATH_INFO: `/v3/users/3fe5f3de75464458b40063b80856cb0f` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:18.920251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe5f3de75464458b40063b80856cb0f)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:18.921776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:18.977939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe5f3de75464458b40063b80856cb0f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:18.997135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe5f3de75464458b40063b80856cb0f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.006662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe5f3de75464458b40063b80856cb0f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.007421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe5f3de75464458b40063b80856cb0f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.008073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe5f3de75464458b40063b80856cb0f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.025821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 977/1954] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:18 2022] DELETE /identity/v3/users/3fe5f3de75464458b40063b80856cb0f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:19.030201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:19.030492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:19.030770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:19.049906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:19.051127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.116878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 978/1955] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:19.117248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:19.119146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.129069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:19.130613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.134495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e5b13-358d-4a95-ac62-a41bbdbd2b26 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:19.140389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:19.141902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:19.142476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:19.143202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] PATH_INFO: `/v3/projects/fff4a4452f6444eeb70ad4f0db32efda` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:19.145935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff4a4452f6444eeb70ad4f0db32efda)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:19.147852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:19.190089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb718968-af41-439f-8bf4-c8a7296b54f9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:19.195427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff4a4452f6444eeb70ad4f0db32efda', 'request_id': 'req-a20a2c94-1089-4b98-ae62-af67028a26df'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.201895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff4a4452f6444eeb70ad4f0db32efda', 'request_id': 'req-a20a2c94-1089-4b98-ae62-af67028a26df'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.209975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff4a4452f6444eeb70ad4f0db32efda', 'request_id': 'req-a20a2c94-1089-4b98-ae62-af67028a26df'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.222026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] The token cache is being invalidate because project fff4a4452f6444eeb70ad4f0db32efda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:19.222311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.222956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.224687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 978/1956] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:19 2022] DELETE /identity/v3/projects/fff4a4452f6444eeb70ad4f0db32efda => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:19.228662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:19.229982 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.242869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:19.243530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:19.243811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:19.244122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:19.245917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.259917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:19.264348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:19.305401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a2c94-1089-4b98-ae62-af67028a26df admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:19.310339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:19.311281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:19.311560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:19.311829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] PATH_INFO: `/v3/users/183d3a89db1347b597894074686a0798` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:19.312180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:19.313055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=183d3a89db1347b597894074686a0798)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:19.313652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:19.314954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:19.323182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 979/1957] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:19.363114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '183d3a89db1347b597894074686a0798'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.373097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '183d3a89db1347b597894074686a0798'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.381176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '183d3a89db1347b597894074686a0798'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.381943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '183d3a89db1347b597894074686a0798'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.382648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '183d3a89db1347b597894074686a0798'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.401217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 979/1958] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:19 2022] DELETE /identity/v3/users/183d3a89db1347b597894074686a0798 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:19.641180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:19.642905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.666322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0ea27-c784-4ff8-91ea-f9b04ff271d6 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:19.761143 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:19.762032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:19.762371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:19.762740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] PATH_INFO: `/v3/projects/dcdc01f666a64a05a7e80dc0c8335b41` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:19.764012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcdc01f666a64a05a7e80dc0c8335b41)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:19.766032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:19.788988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:19.790484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.815512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692822df-8e3b-452b-8117-74e6fe045e2f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:19.834424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcdc01f666a64a05a7e80dc0c8335b41', 'request_id': 'req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.846489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcdc01f666a64a05a7e80dc0c8335b41', 'request_id': 'req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.854043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcdc01f666a64a05a7e80dc0c8335b41', 'request_id': 'req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.866687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] The token cache is being invalidate because project dcdc01f666a64a05a7e80dc0c8335b41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:19.866985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.867658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.869268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 980/1959] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:19 2022] DELETE /identity/v3/projects/dcdc01f666a64a05a7e80dc0c8335b41 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:19.873104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:19.874655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:19.893219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:19.894101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:19.894382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:19.894653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] PATH_INFO: `/v3/projects/22ff62eefc054e00ae80f8312b664744` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:19.895872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ff62eefc054e00ae80f8312b664744)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:19.899030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:19.934953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:19.960812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb30fa-9c12-4531-b2f5-1fe0bff3ba8b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:19.967516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:19.969538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ff62eefc054e00ae80f8312b664744', 'request_id': 'req-3736a32e-a095-4848-ba02-fc36e4951dd4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:19.983786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ff62eefc054e00ae80f8312b664744', 'request_id': 'req-3736a32e-a095-4848-ba02-fc36e4951dd4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.007541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ff62eefc054e00ae80f8312b664744', 'request_id': 'req-3736a32e-a095-4848-ba02-fc36e4951dd4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.037742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] The token cache is being invalidate because project 22ff62eefc054e00ae80f8312b664744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:20.038050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.040022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3736a32e-a095-4848-ba02-fc36e4951dd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.041402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.042880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.043577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.044043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] PATH_INFO: `/v3/users/0bd62767cde9495b9651e177b53d5f1d` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.048635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bd62767cde9495b9651e177b53d5f1d)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.048808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 980/1960] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:19 2022] DELETE /identity/v3/projects/22ff62eefc054e00ae80f8312b664744 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:20.048894 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.050507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3924d8-42d9-46d0-bbf7-9ed531050359 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.050806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3924d8-42d9-46d0-bbf7-9ed531050359 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.051092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3924d8-42d9-46d0-bbf7-9ed531050359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.074753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fa3924d8-42d9-46d0-bbf7-9ed531050359 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:20.076098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fa3924d8-42d9-46d0-bbf7-9ed531050359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:20.088434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 981/1961] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:20.097040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102e8f8-0ca4-4fec-be2c-cf8b038e8639 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.097502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102e8f8-0ca4-4fec-be2c-cf8b038e8639 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.098030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102e8f8-0ca4-4fec-be2c-cf8b038e8639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.121688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-5102e8f8-0ca4-4fec-be2c-cf8b038e8639 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:20.123109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5102e8f8-0ca4-4fec-be2c-cf8b038e8639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:20.125010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bd62767cde9495b9651e177b53d5f1d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.126133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bd62767cde9495b9651e177b53d5f1d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.132587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 982/1962] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:20.137491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102e8f8-0ca4-4fec-be2c-cf8b038e8639 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.140419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bd62767cde9495b9651e177b53d5f1d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.150395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bd62767cde9495b9651e177b53d5f1d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.151912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bd62767cde9495b9651e177b53d5f1d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.154621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102e8f8-0ca4-4fec-be2c-cf8b038e8639 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.160380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.161174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.161548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.161947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.163068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:20.163270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.165113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.179305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 981/1963] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:19 2022] DELETE /identity/v3/users/0bd62767cde9495b9651e177b53d5f1d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:20.183483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 983/1964] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:20.185050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.262960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3562b-c4ce-48fd-b2b9-bae0276a565a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.268308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26391676-d746-4072-9733-7ae511766725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.268955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26391676-d746-4072-9733-7ae511766725 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.269237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26391676-d746-4072-9733-7ae511766725 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.269521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26391676-d746-4072-9733-7ae511766725 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.270227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26391676-d746-4072-9733-7ae511766725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.272188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26391676-d746-4072-9733-7ae511766725 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.287500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:20.307891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 982/1965] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:20.312965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.328170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6a283-842b-4197-b2c1-de0c3440cb06 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.400648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.401488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.401791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.402000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.402694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.404733 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.427000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 984/1966] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:20.431309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26391676-d746-4072-9733-7ae511766725 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.442016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26391676-d746-4072-9733-7ae511766725 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.447682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.448704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.449014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.449328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.450352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.452219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.467554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 983/1967] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:20.468865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.474942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.476991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:20.484237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea37a8-f466-4921-a00b-dbedf1f99e61 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.490870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.491547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.491834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.492157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.492644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759067cd-2bd3-4c56-8f31-b7ae532440be admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.492884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.494508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.499222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.500139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.500428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.500962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] PATH_INFO: `/v3/projects/fc766a7e580c431ca9fc3168d0bc62ab` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.502551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc766a7e580c431ca9fc3168d0bc62ab)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.504520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.506712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 985/1968] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:20.510707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.522701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e922ef-e1bf-40e2-8139-f06cfb999e2e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.527453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067aad33-6bed-40a7-be14-932eb35f74a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.528448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aad33-6bed-40a7-be14-932eb35f74a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.528729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aad33-6bed-40a7-be14-932eb35f74a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.529005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067aad33-6bed-40a7-be14-932eb35f74a3 admin admin] PATH_INFO: `/v3/projects/5bd6d9d01cf64b7e8f41c7f8b0b5c937/users/323eca9c32b8455b8d1deef78050d9be/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.544584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aad33-6bed-40a7-be14-932eb35f74a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd6d9d01cf64b7e8f41c7f8b0b5c937, user_id=323eca9c32b8455b8d1deef78050d9be, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.547349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067aad33-6bed-40a7-be14-932eb35f74a3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.561323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc766a7e580c431ca9fc3168d0bc62ab', 'request_id': 'req-6d4717d8-4b8a-441a-a073-b8644488632b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.564771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 986/1969] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:20 2022] PUT /identity/v3/projects/5bd6d9d01cf64b7e8f41c7f8b0b5c937/users/323eca9c32b8455b8d1deef78050d9be/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:20.569858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc766a7e580c431ca9fc3168d0bc62ab', 'request_id': 'req-6d4717d8-4b8a-441a-a073-b8644488632b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.575397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce71e7-9110-4f60-9e1c-2bc835b1fa89 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.575688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce71e7-9110-4f60-9e1c-2bc835b1fa89 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.575962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce71e7-9110-4f60-9e1c-2bc835b1fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.577784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc766a7e580c431ca9fc3168d0bc62ab', 'request_id': 'req-6d4717d8-4b8a-441a-a073-b8644488632b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.588936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] The token cache is being invalidate because project fc766a7e580c431ca9fc3168d0bc62ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:20.589221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.589880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-6d4717d8-4b8a-441a-a073-b8644488632b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:20.591702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 984/1970] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:20 2022] DELETE /identity/v3/projects/fc766a7e580c431ca9fc3168d0bc62ab => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:20.596246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.596605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.596883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.603975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-76ce71e7-9110-4f60-9e1c-2bc835b1fa89 None None] MFA Rules not processed for user `323eca9c32b8455b8d1deef78050d9be`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:20.605225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-76ce71e7-9110-4f60-9e1c-2bc835b1fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:20.614762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:20.616312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:20.662093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:20.673509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 987/1971] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:20.680216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 985/1972] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:20.688185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76054-b9ec-4a44-ba5f-8d61976f3711 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.688477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76054-b9ec-4a44-ba5f-8d61976f3711 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.688822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c76054-b9ec-4a44-ba5f-8d61976f3711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.694213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.700873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:20.713056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-b8c76054-b9ec-4a44-ba5f-8d61976f3711 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:20.714249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b8c76054-b9ec-4a44-ba5f-8d61976f3711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:20.724457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 988/1973] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:20.727305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c76054-b9ec-4a44-ba5f-8d61976f3711 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.738295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c76054-b9ec-4a44-ba5f-8d61976f3711 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.744359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.745260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.745648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.746190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.747073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:20.747454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.749233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.762217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 989/1974] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:51:20.766798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.778330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f3fc1-7f37-4f52-bbb2-e5ee7fc1dbe6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.779863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ab678-1629-40bb-95a0-f797d74d76ba None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.784600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.785275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.785529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.785934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.790028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79affb71-6ae2-48df-af10-6c2f06874b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.790716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79affb71-6ae2-48df-af10-6c2f06874b2d admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.790998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79affb71-6ae2-48df-af10-6c2f06874b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.791274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79affb71-6ae2-48df-af10-6c2f06874b2d admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.791977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79affb71-6ae2-48df-af10-6c2f06874b2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.794043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79affb71-6ae2-48df-af10-6c2f06874b2d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.796118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.797693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.808261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 986/1975] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:20.827876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 990/1976] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:20.832240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.847543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364ab24-4873-4e6d-8777-7c91a488f01c service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.856265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:20.925632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7ea29-df47-4f40-89f3-7727d5d51cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.926426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ea29-df47-4f40-89f3-7727d5d51cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.926806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ea29-df47-4f40-89f3-7727d5d51cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.927169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ea29-df47-4f40-89f3-7727d5d51cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.928476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7ea29-df47-4f40-89f3-7727d5d51cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.930344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7ea29-df47-4f40-89f3-7727d5d51cb7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:20.951071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f46e5-2604-4712-95c8-0f0278425b4a None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.951414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f46e5-2604-4712-95c8-0f0278425b4a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.951900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f46e5-2604-4712-95c8-0f0278425b4a None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.953845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 991/1977] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:51:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:51:20.962320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 987/1978] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:20.966973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f46e5-2604-4712-95c8-0f0278425b4a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:20.982686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f46e5-2604-4712-95c8-0f0278425b4a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:20.989423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:20.990726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.991171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.991566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.991934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:20.992619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:20.992864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:20.993842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:20.995281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.013131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 992/1979] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:21.015539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.021773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:21.023512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:21.028904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b8f68-a686-4822-934d-d4a3ca6d5fdb admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.035424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.036304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.036705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.037404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.038078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.040176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.054246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 993/1980] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:21.067874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.092799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e71c8-a13c-4562-8487-d3aba72ff6f3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.124210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322d999-3fee-4a5b-bfbc-48f395564790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.125491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322d999-3fee-4a5b-bfbc-48f395564790 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.125975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322d999-3fee-4a5b-bfbc-48f395564790 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.126406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322d999-3fee-4a5b-bfbc-48f395564790 admin admin] PATH_INFO: `/v3/projects/24bfd2ba81104fe48a55f57942474bcd/users/c46a0a0e45694029934464c2cda9d5c9/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.141097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 988/1981] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:51:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:21.146199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322d999-3fee-4a5b-bfbc-48f395564790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24bfd2ba81104fe48a55f57942474bcd, user_id=c46a0a0e45694029934464c2cda9d5c9, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.149698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322d999-3fee-4a5b-bfbc-48f395564790 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.171026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.173224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:21.176573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 994/1982] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:21 2022] PUT /identity/v3/projects/24bfd2ba81104fe48a55f57942474bcd/users/c46a0a0e45694029934464c2cda9d5c9/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:21.186240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5a036-6fca-4534-b413-1c64522bb72d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.187553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5a036-6fca-4534-b413-1c64522bb72d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.187553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5a036-6fca-4534-b413-1c64522bb72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.195699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f582f67-88df-4e72-90af-aa85e8bcdbd1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.244886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c7a5a036-6fca-4534-b413-1c64522bb72d None None] MFA Rules not processed for user `c46a0a0e45694029934464c2cda9d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:21.248419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c7a5a036-6fca-4534-b413-1c64522bb72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:21.346935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.347827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.348211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.349729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.360357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.362320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.376864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 995/1983] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:21.382195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5a036-6fca-4534-b413-1c64522bb72d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.391045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 989/1984] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:21.465529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:21.478148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5a036-6fca-4534-b413-1c64522bb72d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.483743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.484784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.484784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.484981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.485738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.487727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.521674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 996/1985] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:21 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:21.526196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.541863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7deb1-648a-46f2-ad75-64f7c9953feb service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.553559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:21.598718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.599389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.599671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.599947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.600721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.602644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.627873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 990/1986] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:21 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:21.632163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.642799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4477055-6a99-4f3c-84fc-ce5c58db0a7f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.648090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.648748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.649030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.649331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.650053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.651655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.662700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 997/1987] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:21.667090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.677471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3310174-6767-486d-87a5-94e8b055b7af admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.682472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.683353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.683632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.683911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] PATH_INFO: `/v3/projects/e972d2524081496b9ac83a85e4b4c751/users/54eeca147da941168d270ecc07e17b03/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.702264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e972d2524081496b9ac83a85e4b4c751, user_id=54eeca147da941168d270ecc07e17b03, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.705047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.719864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 991/1988] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:21 2022] PUT /identity/v3/projects/e972d2524081496b9ac83a85e4b4c751/users/54eeca147da941168d270ecc07e17b03/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:21.723897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.737977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16c71c-36c0-4024-922b-a1bf7e755405 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.797067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1dc5ca-dd1b-4511-b822-1d53f97827e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.801681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dc5ca-dd1b-4511-b822-1d53f97827e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.802194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dc5ca-dd1b-4511-b822-1d53f97827e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.803857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dc5ca-dd1b-4511-b822-1d53f97827e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.806823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1dc5ca-dd1b-4511-b822-1d53f97827e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.808912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1dc5ca-dd1b-4511-b822-1d53f97827e3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.820188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 998/1989] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:21.824681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:21.835838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e285e-89e0-40a2-af0b-8b6edbaabc8e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:21.841799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:21.843084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.843455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.843824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] PATH_INFO: `/v3/projects/e972d2524081496b9ac83a85e4b4c751/users/54eeca147da941168d270ecc07e17b03/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.847369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e972d2524081496b9ac83a85e4b4c751, user_id=54eeca147da941168d270ecc07e17b03, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:21.851866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:21.864533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 992/1990] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:21 2022] PUT /identity/v3/projects/e972d2524081496b9ac83a85e4b4c751/users/54eeca147da941168d270ecc07e17b03/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:21.875328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:21.875619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:21.875900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:21.903364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] MFA Rules not processed for user `54eeca147da941168d270ecc07e17b03`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:21.904604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:21.984474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 999/1991] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:21.997657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:22.060360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b773272-abfa-45c5-9863-e732657ba02a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:22.064258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ce2e5-9315-466e-bbdd-7a1143f73c71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:22.064931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce2e5-9315-466e-bbdd-7a1143f73c71 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:22.065231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce2e5-9315-466e-bbdd-7a1143f73c71 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:22.065506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ce2e5-9315-466e-bbdd-7a1143f73c71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:22.127773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ce2e5-9315-466e-bbdd-7a1143f73c71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:22.129344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950ce2e5-9315-466e-bbdd-7a1143f73c71 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:22.140039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 993/1992] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:23.016689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:23.018257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:23.078324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:23.092744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf19af7-0b1d-4484-bc1e-64351d54ce2b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:23.097956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:23.098841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:23.099118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:23.099387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] PATH_INFO: `/v3/users/610d591a6fd24ed6980abc68416326fc` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:23.100796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=610d591a6fd24ed6980abc68416326fc)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:23.102727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:23.152250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '610d591a6fd24ed6980abc68416326fc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:23.160429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '610d591a6fd24ed6980abc68416326fc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:23.167419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '610d591a6fd24ed6980abc68416326fc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:23.168091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '610d591a6fd24ed6980abc68416326fc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:23.168723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '610d591a6fd24ed6980abc68416326fc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:23.187083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1000/1993] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:23 2022] DELETE /identity/v3/users/610d591a6fd24ed6980abc68416326fc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:23.191861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:23.192349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:23.192459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:23.209930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:23.211149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:23.228327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:23.273918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 994/1994] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:23.284720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:23.286264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:23.345317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d314d13e-422a-477a-acf0-36877cf65506 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:23.350184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:23.350828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:23.351106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:23.351374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:23.361494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:23.362930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:23.373124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1001/1995] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:23.914792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:23.916297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:23.936360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239dbc8b-beae-49cc-b197-95569c852bb7 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:23.959809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:23.960783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:23.961120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:23.961927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] PATH_INFO: `/v3/projects/ce2f5182f6554fa7a1c0c651c172ca8c` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:23.967563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2f5182f6554fa7a1c0c651c172ca8c)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:23.969591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:24.030616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2f5182f6554fa7a1c0c651c172ca8c', 'request_id': 'req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.040637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2f5182f6554fa7a1c0c651c172ca8c', 'request_id': 'req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.048438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2f5182f6554fa7a1c0c651c172ca8c', 'request_id': 'req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.060331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] The token cache is being invalidate because project ce2f5182f6554fa7a1c0c651c172ca8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:24.060699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.061412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.063045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 995/1996] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:23 2022] DELETE /identity/v3/projects/ce2f5182f6554fa7a1c0c651c172ca8c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:24.067251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:24.077342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a5e3-e1e5-4603-b8cd-560227977dbd service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:24.137877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:24.138698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:24.138979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:24.139249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] PATH_INFO: `/v3/users/c9bdf4cf9c474a199592c3ad4c2d59b5` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:24.140459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9bdf4cf9c474a199592c3ad4c2d59b5)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:24.142378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:24.155707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:24.192773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9bdf4cf9c474a199592c3ad4c2d59b5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.201936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9bdf4cf9c474a199592c3ad4c2d59b5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.209478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9bdf4cf9c474a199592c3ad4c2d59b5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.210130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9bdf4cf9c474a199592c3ad4c2d59b5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.210754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-af7ab277-50db-4fa4-b88f-b0f6ddce33e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9bdf4cf9c474a199592c3ad4c2d59b5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.228228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1002/1997] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:24 2022] DELETE /identity/v3/users/c9bdf4cf9c474a199592c3ad4c2d59b5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:24.511348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:24.512930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:24.529002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1761d9-7cf8-412e-bbb8-6abf41fc4289 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:24.596099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:24.597442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:24.597765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:24.598150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] PATH_INFO: `/v3/projects/155a1408a72e404f99a059ce6748d9e5` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:24.600148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155a1408a72e404f99a059ce6748d9e5)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:24.603502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:24.653647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:24.654948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155a1408a72e404f99a059ce6748d9e5', 'request_id': 'req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.664596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155a1408a72e404f99a059ce6748d9e5', 'request_id': 'req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.672977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155a1408a72e404f99a059ce6748d9e5', 'request_id': 'req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.684823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] The token cache is being invalidate because project 155a1408a72e404f99a059ce6748d9e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:24.685116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.685862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0be6b6bd-f3c6-4e22-afc6-1c474bafe533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:24.687741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 996/1998] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:24 2022] DELETE /identity/v3/projects/155a1408a72e404f99a059ce6748d9e5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:24.693337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccad0ff-73b1-45c2-b825-20d0a0a3fc8b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:24.693337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccad0ff-73b1-45c2-b825-20d0a0a3fc8b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:24.693666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccad0ff-73b1-45c2-b825-20d0a0a3fc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:24.713025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fccad0ff-73b1-45c2-b825-20d0a0a3fc8b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:24.714242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fccad0ff-73b1-45c2-b825-20d0a0a3fc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:24.789972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1003/1999] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:24.797677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abad3-1c81-407a-8041-4069ffc4b614 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:24.797991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abad3-1c81-407a-8041-4069ffc4b614 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:24.798274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299abad3-1c81-407a-8041-4069ffc4b614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:24.817385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-299abad3-1c81-407a-8041-4069ffc4b614 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:24.818603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-299abad3-1c81-407a-8041-4069ffc4b614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:24.829519 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 997/2000] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:24.831287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccad0ff-73b1-45c2-b825-20d0a0a3fc8b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:24.841200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccad0ff-73b1-45c2-b825-20d0a0a3fc8b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:24.845829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:24.846496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:24.846876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:24.847053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:24.848238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:24.848603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:24.850215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:24.867582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1004/2001] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:24.870300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abad3-1c81-407a-8041-4069ffc4b614 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:24.880504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299abad3-1c81-407a-8041-4069ffc4b614 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:24.885446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:24.886124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:24.886407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:24.886686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:24.887481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:24.889435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:24.918290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 998/2002] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:24 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:24.922568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:24.932831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c378c65-9811-40de-b312-b0ebced3bdfc admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.001177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.001912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.002199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.002469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.003166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.005193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.027022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1005/2003] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:24 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:25.030937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.041533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998edb7-49bd-467e-b03c-7ac431818d8c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.046840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.047620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.047781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.048088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.048822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.050590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.062573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 999/2004] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:25.066612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.076931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd842bc1-9c52-40ec-b50c-2ef5130e0979 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.081668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a37ecb-b506-4403-a0b5-9335142d371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.082330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a37ecb-b506-4403-a0b5-9335142d371d admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.082612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a37ecb-b506-4403-a0b5-9335142d371d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.082909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a37ecb-b506-4403-a0b5-9335142d371d admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.083640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a37ecb-b506-4403-a0b5-9335142d371d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.085292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a37ecb-b506-4403-a0b5-9335142d371d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.087361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:25.099339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1006/2005] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:25.101680 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.112036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b11b4-c7a1-4e50-8d21-cfa225ec1810 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.117907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.118800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.119082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.119348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] PATH_INFO: `/v3/projects/2f0e8600ff2a4bb98f2ee3f1ea6bafda/users/282c6a8f8f0046bcb0383b06b3dd05ee/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.134781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0e8600ff2a4bb98f2ee3f1ea6bafda, user_id=282c6a8f8f0046bcb0383b06b3dd05ee, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.137616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.152374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1000/2006] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:25 2022] PUT /identity/v3/projects/2f0e8600ff2a4bb98f2ee3f1ea6bafda/users/282c6a8f8f0046bcb0383b06b3dd05ee/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:25.162698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f076bd-a9ee-4f1a-87e5-1a6c80574a6d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.162991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f076bd-a9ee-4f1a-87e5-1a6c80574a6d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.163271 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f076bd-a9ee-4f1a-87e5-1a6c80574a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.192731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c1f076bd-a9ee-4f1a-87e5-1a6c80574a6d None None] MFA Rules not processed for user `282c6a8f8f0046bcb0383b06b3dd05ee`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:25.193965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c1f076bd-a9ee-4f1a-87e5-1a6c80574a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:25.259608 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1007/2007] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:25.264235 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.327263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92771e0-bf92-4a22-97f2-eabd81902006 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.332671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.333337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.333615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.333887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.334604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.336702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.365791 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1001/2008] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:25 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:25.370001 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f076bd-a9ee-4f1a-87e5-1a6c80574a6d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.381043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f076bd-a9ee-4f1a-87e5-1a6c80574a6d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.440432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.441023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.441360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.441639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.442333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.444341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.464282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1008/2009] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:25 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:25.468888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.479533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b6276-f67b-42ff-80c7-ce770d85c320 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.484724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.485413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.485695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.485953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.486705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.488410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.499424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1002/2010] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:25.503533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.513685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b05921-3366-4d8d-afc3-396f9c338756 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.518767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.519648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.519932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.520237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] PATH_INFO: `/v3/projects/81cc65681ce749c28e5a56a5b71dc82a/users/5e0c2a43a46946708150bbf1811e304a/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.536522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81cc65681ce749c28e5a56a5b71dc82a, user_id=5e0c2a43a46946708150bbf1811e304a, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.539169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.553691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1009/2011] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:25 2022] PUT /identity/v3/projects/81cc65681ce749c28e5a56a5b71dc82a/users/5e0c2a43a46946708150bbf1811e304a/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:25.557560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.567789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53aa96d-148f-455d-acf1-4071426228a7 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.611392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:25.628013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90e5e5-c861-494b-9b94-4352cc2517a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.628742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90e5e5-c861-494b-9b94-4352cc2517a7 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.629021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90e5e5-c861-494b-9b94-4352cc2517a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.629293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90e5e5-c861-494b-9b94-4352cc2517a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.630022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90e5e5-c861-494b-9b94-4352cc2517a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.631650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90e5e5-c861-494b-9b94-4352cc2517a7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.642314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1003/2012] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:25.646810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.657886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db86a8-3b92-40a7-9874-59ec2c4123f6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.664681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.665573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.665851 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.666121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] PATH_INFO: `/v3/projects/81cc65681ce749c28e5a56a5b71dc82a/users/5e0c2a43a46946708150bbf1811e304a/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.668351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81cc65681ce749c28e5a56a5b71dc82a, user_id=5e0c2a43a46946708150bbf1811e304a, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.671025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.681903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1010/2013] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:25 2022] PUT /identity/v3/projects/81cc65681ce749c28e5a56a5b71dc82a/users/5e0c2a43a46946708150bbf1811e304a/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:25.693244 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.693523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.693819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.723459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] MFA Rules not processed for user `5e0c2a43a46946708150bbf1811e304a`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:25.724751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:25.805050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1004/2014] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:25.817061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:25.819519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:25.866639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:25.883624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3be12-07e7-4c26-81cd-a0e1314fb3c3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:25.887883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:25.888586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:25.888586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:25.888819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:25.902138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:25.904829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:25.921041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1011/2015] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:26.011431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:26.013068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:26.025930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132b10c-25a0-482d-9eba-2c5649d30f9c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:26.029786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20784de-ee02-4b79-b5fd-c090a3a2927a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:26.030416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20784de-ee02-4b79-b5fd-c090a3a2927a service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:26.030724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20784de-ee02-4b79-b5fd-c090a3a2927a service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:26.030963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20784de-ee02-4b79-b5fd-c090a3a2927a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:26.113923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20784de-ee02-4b79-b5fd-c090a3a2927a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:26.116338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20784de-ee02-4b79-b5fd-c090a3a2927a service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:26.134021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1005/2016] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:26.490803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:26.555803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e13f1-492a-4c0a-86f7-5fd7937c5197 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:26.562178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:26.563126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:26.563918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:26.564279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] PATH_INFO: `/v3/users/282c6a8f8f0046bcb0383b06b3dd05ee` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:26.565976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=282c6a8f8f0046bcb0383b06b3dd05ee)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:26.568499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:26.621025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '282c6a8f8f0046bcb0383b06b3dd05ee'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:26.629057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '282c6a8f8f0046bcb0383b06b3dd05ee'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:26.636437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '282c6a8f8f0046bcb0383b06b3dd05ee'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:26.637187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '282c6a8f8f0046bcb0383b06b3dd05ee'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:26.637881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '282c6a8f8f0046bcb0383b06b3dd05ee'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:26.654904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1012/2017] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:26 2022] DELETE /identity/v3/users/282c6a8f8f0046bcb0383b06b3dd05ee => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:26.659574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:26.660075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:26.660189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:26.677297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:26.678532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:26.745468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1006/2018] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:26.758843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:26.760435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:26.827589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59d57-322b-4685-b59f-27081643fc90 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:26.834021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:26.834693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:26.834977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:26.835249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:26.847879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:26.849581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:26.865732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1013/2019] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:27.668407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:27.670505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:27.690985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0bee6-2190-4675-b2ae-493e50b4cc7e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:27.706394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:27.707249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:27.707428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:27.708028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] PATH_INFO: `/v3/projects/2f0e8600ff2a4bb98f2ee3f1ea6bafda` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:27.711817 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0e8600ff2a4bb98f2ee3f1ea6bafda)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:27.714089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:27.718077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:27.807477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0e8600ff2a4bb98f2ee3f1ea6bafda', 'request_id': 'req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:27.827071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0e8600ff2a4bb98f2ee3f1ea6bafda', 'request_id': 'req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:27.834539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0e8600ff2a4bb98f2ee3f1ea6bafda', 'request_id': 'req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:27.847924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] The token cache is being invalidate because project 2f0e8600ff2a4bb98f2ee3f1ea6bafda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:27.848331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:27.849056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:27.851199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1007/2020] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:27 2022] DELETE /identity/v3/projects/2f0e8600ff2a4bb98f2ee3f1ea6bafda => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:27.855635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:27.863686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:27.869679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567b7bf-7ef5-4325-9ede-ac3903ff8151 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:27.956416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:27.957328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:27.957984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:27.957984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] PATH_INFO: `/v3/users/5e0c2a43a46946708150bbf1811e304a` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:27.959307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0c2a43a46946708150bbf1811e304a)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:27.961676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:28.029115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0c2a43a46946708150bbf1811e304a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.038683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0c2a43a46946708150bbf1811e304a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.046209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0c2a43a46946708150bbf1811e304a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.046875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0c2a43a46946708150bbf1811e304a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.047478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-69306d91-4a56-4b10-9379-b00da94bb50a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0c2a43a46946708150bbf1811e304a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.065795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1014/2021] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:27 2022] DELETE /identity/v3/users/5e0c2a43a46946708150bbf1811e304a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:28.361080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:28.363476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:28.381075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71e335-a2ca-4f5e-9d64-541b7d3827cc admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:28.450453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:28.451409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.451692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.451962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] PATH_INFO: `/v3/projects/81cc65681ce749c28e5a56a5b71dc82a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.453174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] RBAC: Authorizing `identity:delete_project(project_id=81cc65681ce749c28e5a56a5b71dc82a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:28.455082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:28.508342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81cc65681ce749c28e5a56a5b71dc82a', 'request_id': 'req-a7eefe46-b10b-4eb6-b518-082e389fa24e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.518522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81cc65681ce749c28e5a56a5b71dc82a', 'request_id': 'req-a7eefe46-b10b-4eb6-b518-082e389fa24e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.526082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81cc65681ce749c28e5a56a5b71dc82a', 'request_id': 'req-a7eefe46-b10b-4eb6-b518-082e389fa24e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.540852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] The token cache is being invalidate because project 81cc65681ce749c28e5a56a5b71dc82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:28.541003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.541656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a7eefe46-b10b-4eb6-b518-082e389fa24e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:28.543314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1008/2022] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:28 2022] DELETE /identity/v3/projects/81cc65681ce749c28e5a56a5b71dc82a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:28.548308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580e85-7ffe-4f5c-adb8-cb416bc31d8f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.548590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580e85-7ffe-4f5c-adb8-cb416bc31d8f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.548889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57580e85-7ffe-4f5c-adb8-cb416bc31d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.569936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-57580e85-7ffe-4f5c-adb8-cb416bc31d8f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:28.571138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-57580e85-7ffe-4f5c-adb8-cb416bc31d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:28.637923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1015/2023] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:28.645928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e773335-34b3-4deb-a65e-4310d5096adc None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.646217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e773335-34b3-4deb-a65e-4310d5096adc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.646496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e773335-34b3-4deb-a65e-4310d5096adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.666593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8e773335-34b3-4deb-a65e-4310d5096adc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:28.667775 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8e773335-34b3-4deb-a65e-4310d5096adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:28.676451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1009/2024] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:28.681205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580e85-7ffe-4f5c-adb8-cb416bc31d8f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:28.694486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57580e85-7ffe-4f5c-adb8-cb416bc31d8f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:28.709357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:28.710066 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.710347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.710626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.714270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:28.714592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:28.718789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:28.747859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1016/2025] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:28.752728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e773335-34b3-4deb-a65e-4310d5096adc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:28.764214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e773335-34b3-4deb-a65e-4310d5096adc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:28.769529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:28.770269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.770755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.771199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.771975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:28.773950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:28.785553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:28.808526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1010/2026] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:28 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:28.810996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:28.824389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0586a-3f92-4fbd-9722-27ec9c13e327 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:28.833465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:28.845974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:28.847795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:28.900801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:28.907898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.916404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.916779 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.917490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:28.919532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:28.935527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce5446-1345-4df7-9993-412d92fd7e29 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:28.939306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:28.940278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.940559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.940860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] PATH_INFO: `/v3/users/2a43fb1726c740d18918e3ca1835b036` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.942072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a43fb1726c740d18918e3ca1835b036)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:28.944740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1017/2027] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:28 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:28.945260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:28.961588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:28.983890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57f931-30c7-42b8-91c3-9ddbbeb66736 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:28.991227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:28.992031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:28.992317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:28.992593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:28.993422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:28.995256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:29.007406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1018/2028] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:29.010909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a43fb1726c740d18918e3ca1835b036'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.011252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:29.012317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a43fb1726c740d18918e3ca1835b036'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.028136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e392fa1-cf92-4f98-8fe1-3692d696c515 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:29.032873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a43fb1726c740d18918e3ca1835b036'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.039545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:29.040402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.040700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.040977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.042044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:29.043354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:29.051847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a43fb1726c740d18918e3ca1835b036'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.052643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-35a8944e-d2aa-4c72-92df-0f4de523269f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a43fb1726c740d18918e3ca1835b036'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.057358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1019/2029] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:29.065864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:29.067754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:29.071407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1011/2030] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:28 2022] DELETE /identity/v3/users/2a43fb1726c740d18918e3ca1835b036 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:29.076618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.077044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.077387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.093651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8f1752-0daf-4b60-b10c-2d51cd9a20da admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:29.102874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:29.104965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:29.162704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7e868-2a99-4be0-b059-8e0957140c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:29.163676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7e868-2a99-4be0-b059-8e0957140c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.164399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7e868-2a99-4be0-b059-8e0957140c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.164704 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7e868-2a99-4be0-b059-8e0957140c4f admin admin] PATH_INFO: `/v3/projects/cd4585e1e7524dd78f3cf187684ee2ba/users/17123d3e5fe945bbb1cb835f154965d4/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.188817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7e868-2a99-4be0-b059-8e0957140c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4585e1e7524dd78f3cf187684ee2ba, user_id=17123d3e5fe945bbb1cb835f154965d4, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:29.192939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7e868-2a99-4be0-b059-8e0957140c4f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:29.206206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1012/2031] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:29.215596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1020/2032] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:29 2022] PUT /identity/v3/projects/cd4585e1e7524dd78f3cf187684ee2ba/users/17123d3e5fe945bbb1cb835f154965d4/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:29.219233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:29.220380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:29.220965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5121-71d1-47dd-838a-ab403b9ec100 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.221252 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5121-71d1-47dd-838a-ab403b9ec100 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.221827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5121-71d1-47dd-838a-ab403b9ec100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.241199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8b4f5121-71d1-47dd-838a-ab403b9ec100 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:29.242717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8b4f5121-71d1-47dd-838a-ab403b9ec100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:29.243837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:29.285292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44407c9b-340e-4b21-9b69-a26be99d1112 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:29.304118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1021/2033] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:29.313588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f5121-71d1-47dd-838a-ab403b9ec100 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:29.317513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:29.333986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:29.334639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.334920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.335193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.344788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:29.346231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:29.355724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1013/2034] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:29.373660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f5121-71d1-47dd-838a-ab403b9ec100 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:29.377778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86fc5e-ef2b-4065-b57d-ea870868b20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:29.378432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86fc5e-ef2b-4065-b57d-ea870868b20e service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.378716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86fc5e-ef2b-4065-b57d-ea870868b20e service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.378992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86fc5e-ef2b-4065-b57d-ea870868b20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.388841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86fc5e-ef2b-4065-b57d-ea870868b20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:29.390276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86fc5e-ef2b-4065-b57d-ea870868b20e service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:29.400352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1022/2035] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:29.636457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:29.638383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:29.652712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c111e-c257-4e61-bca9-e7d9c57f3697 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:29.659467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:29.660593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.660820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.661418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] PATH_INFO: `/v3/users/323eca9c32b8455b8d1deef78050d9be` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.662853 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] RBAC: Authorizing `identity:delete_user(user_id=323eca9c32b8455b8d1deef78050d9be)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:29.664788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:29.726422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '323eca9c32b8455b8d1deef78050d9be'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.727310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '323eca9c32b8455b8d1deef78050d9be'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.745355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '323eca9c32b8455b8d1deef78050d9be'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.756075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '323eca9c32b8455b8d1deef78050d9be'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.757328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '323eca9c32b8455b8d1deef78050d9be'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:29.796162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1014/2036] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:29 2022] DELETE /identity/v3/users/323eca9c32b8455b8d1deef78050d9be => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:29.801886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:29.802501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:29.802994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:29.826814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:29.828362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:29.868420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:29.868420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:29.971184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:30.025514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61532c-6210-4fd1-817c-d8437348079b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:30.027884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1023/2037] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:30.032793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:30.033907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:30.034362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:30.034815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] PATH_INFO: `/v3/projects/fe494de3259b44b1bc8c1265d919a21a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:30.036628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe494de3259b44b1bc8c1265d919a21a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:30.038497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:30.040522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:30.041343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:30.103609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe494de3259b44b1bc8c1265d919a21a', 'request_id': 'req-85ea3389-f199-41e4-a9c0-0106c4f3167c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.118117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe494de3259b44b1bc8c1265d919a21a', 'request_id': 'req-85ea3389-f199-41e4-a9c0-0106c4f3167c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.129211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe494de3259b44b1bc8c1265d919a21a', 'request_id': 'req-85ea3389-f199-41e4-a9c0-0106c4f3167c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.132465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d36a25-1353-4794-b08b-13a066db8a8b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:30.142860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] The token cache is being invalidate because project fe494de3259b44b1bc8c1265d919a21a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:30.143435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.144765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.147166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1015/2038] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:29 2022] DELETE /identity/v3/projects/fe494de3259b44b1bc8c1265d919a21a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:30.148208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:30.151169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:30.208535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:30.209266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:30.209480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:30.209832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:30.211918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:30.231293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:30.273988 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea3389-f199-41e4-a9c0-0106c4f3167c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:30.278689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:30.283679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:30.283679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:30.283679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:30.283679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:30.302379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:30.303931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:30.320222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1024/2039] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:30.321080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:30.324169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:30.345864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb748d4-cc95-4ed6-bae0-ef10654e3699 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:30.352688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:30.354213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:30.354499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:30.354770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] PATH_INFO: `/v3/users/df6d6f5f724443028d68062bc16a4991` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:30.356359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df6d6f5f724443028d68062bc16a4991)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:30.359223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:30.376300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:30.378327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:30.398839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1016/2040] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:30.465437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df6d6f5f724443028d68062bc16a4991'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.477128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df6d6f5f724443028d68062bc16a4991'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.488978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df6d6f5f724443028d68062bc16a4991'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.490117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df6d6f5f724443028d68062bc16a4991'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.491680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df6d6f5f724443028d68062bc16a4991'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:30.528534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1025/2041] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:30 2022] DELETE /identity/v3/users/df6d6f5f724443028d68062bc16a4991 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:31.030282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.032476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.077663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b763c-103b-41c8-b20a-f3ee56987b40 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.115836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:31.180641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.182245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.221926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.222843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.223121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.223456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] PATH_INFO: `/v3/projects/5bd6d9d01cf64b7e8f41c7f8b0b5c937` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.225108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd6d9d01cf64b7e8f41c7f8b0b5c937)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:31.232278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:31.324034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd6d9d01cf64b7e8f41c7f8b0b5c937', 'request_id': 'req-05d95565-f46b-49b1-b578-6378f9468b2a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.328350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b91627-24d9-4461-9461-df52d4751ac3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.335620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd6d9d01cf64b7e8f41c7f8b0b5c937', 'request_id': 'req-05d95565-f46b-49b1-b578-6378f9468b2a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.347450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd6d9d01cf64b7e8f41c7f8b0b5c937', 'request_id': 'req-05d95565-f46b-49b1-b578-6378f9468b2a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.365409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] The token cache is being invalidate because project 5bd6d9d01cf64b7e8f41c7f8b0b5c937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:31.365728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.366295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05d95565-f46b-49b1-b578-6378f9468b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.368355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1017/2042] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:31 2022] DELETE /identity/v3/projects/5bd6d9d01cf64b7e8f41c7f8b0b5c937 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:31.369452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc29a9d-ec98-4784-9049-97907b8d2ba0 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.369733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc29a9d-ec98-4784-9049-97907b8d2ba0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.370003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc29a9d-ec98-4784-9049-97907b8d2ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.379757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:31.404720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6dc29a9d-ec98-4784-9049-97907b8d2ba0 None None] MFA Rules not processed for user `17123d3e5fe945bbb1cb835f154965d4`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:31.406040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6dc29a9d-ec98-4784-9049-97907b8d2ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.419732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.420637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.420919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.421193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] PATH_INFO: `/v3/projects/ab7ff8e8b6774a1484c61bef5208c4bb` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.422384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7ff8e8b6774a1484c61bef5208c4bb)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:31.424358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:31.492496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7ff8e8b6774a1484c61bef5208c4bb', 'request_id': 'req-98eb80e5-6e99-4d45-bb53-926a7b4596ac'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.498246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1018/2043] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:31.499191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7ff8e8b6774a1484c61bef5208c4bb', 'request_id': 'req-98eb80e5-6e99-4d45-bb53-926a7b4596ac'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.499969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea3fd-f9b4-488b-bf30-a9b55a7078f5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.500556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea3fd-f9b4-488b-bf30-a9b55a7078f5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.501013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fea3fd-f9b4-488b-bf30-a9b55a7078f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.509544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7ff8e8b6774a1484c61bef5208c4bb', 'request_id': 'req-98eb80e5-6e99-4d45-bb53-926a7b4596ac'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.521311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-37fea3fd-f9b4-488b-bf30-a9b55a7078f5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:31.522805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-37fea3fd-f9b4-488b-bf30-a9b55a7078f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.526288 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] The token cache is being invalidate because project ab7ff8e8b6774a1484c61bef5208c4bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:31.526706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.527615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:31.529551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1026/2044] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:31 2022] DELETE /identity/v3/projects/ab7ff8e8b6774a1484c61bef5208c4bb => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:31.530605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.532064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.581354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:31.601444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb80e5-6e99-4d45-bb53-926a7b4596ac admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.605205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.605860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.606173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.606449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.608222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1019/2045] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:31.609472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62726e5e-b720-4ea4-8eae-c1563e1ea99a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.609759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62726e5e-b720-4ea4-8eae-c1563e1ea99a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.610197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62726e5e-b720-4ea4-8eae-c1563e1ea99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.617344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.632767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-62726e5e-b720-4ea4-8eae-c1563e1ea99a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:31.633987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-62726e5e-b720-4ea4-8eae-c1563e1ea99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.642523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1020/2046] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:31.643702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f49f-9ae2-4d76-b6be-ba9ab9a6f758 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.644076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f49f-9ae2-4d76-b6be-ba9ab9a6f758 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.644310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288f49f-9ae2-4d76-b6be-ba9ab9a6f758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.662701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b288f49f-9ae2-4d76-b6be-ba9ab9a6f758 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:31.663915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b288f49f-9ae2-4d76-b6be-ba9ab9a6f758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.677336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1021/2047] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:31.680673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bc76e-a5fb-4de8-8114-80a63cb81bbe None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.680673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bc76e-a5fb-4de8-8114-80a63cb81bbe None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.680673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bc76e-a5fb-4de8-8114-80a63cb81bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.699948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-101bc76e-a5fb-4de8-8114-80a63cb81bbe None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:31.701206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-101bc76e-a5fb-4de8-8114-80a63cb81bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:31.706361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:31.707922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:31.710085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1022/2048] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:31.713471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bc76e-a5fb-4de8-8114-80a63cb81bbe None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.723230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bc76e-a5fb-4de8-8114-80a63cb81bbe None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.730071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.730558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1027/2049] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:31.730737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.731057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.731333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.732205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:31.732520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:31.732877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.734277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:31.753306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1023/2050] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:31.754235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4cff6-4eaa-4703-b99c-c8f9f1968b5a service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.754716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.760348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.761647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.762961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.763728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.765911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfe5de-f42e-48d3-89ab-aa77b690b75f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.766241 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:31.767078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:31.769010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:31.771085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.772213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.772213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.772451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.773161 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:31.775330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:31.789806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1028/2051] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:31.794118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.804981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:31.813128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f7c03-5391-4ffd-a172-cc2a01873cde admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.821555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.822718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.823923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.824586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.825734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:31.828160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:31.834527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1024/2052] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:31.839264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.863303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1029/2053] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:31.867781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:31.922244 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa6d48-bba6-44c1-8091-cad1bb728950 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.968897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99985a-73d6-4be3-9f60-beb5801e6ac0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:31.982939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:31.984951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:31.985271 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:31.985552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:31.986287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.016395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.070922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1030/2054] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:32.076182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.096807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ed6aa-0f3f-49b4-8496-0fc3b22f389d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.120718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.125075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.132089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.132228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.132489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.132797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.133106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.133350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.134010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.135915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.139774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.148504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.175744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1031/2055] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:32.177750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.195890 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dc769-e7a6-4a9e-8ffd-9abfb4b85f15 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.206291 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1025/2056] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:31 2022] POST /identity/v3/users => generated 288 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:32.208552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.211297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.213690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.225744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.225744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.225744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.231078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.233446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9200f83-6647-40bc-8ea9-299e5874114a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.245880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.250011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.250290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.250370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.251085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.255724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.262974 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1032/2057] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:32.270359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.292040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1026/2058] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:32.304270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.307919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c303b8-568d-47c2-a12f-0ca7a4b62095 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.327429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.328358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.328658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.328902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] PATH_INFO: `/v3/projects/e2aa384b631244bf87de25b2ff0b8944/users/1eb0bee5b68c447dba83ae8742cf5eb6/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.331624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d370b3-9766-4226-b229-8cb8dc722be4 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.344380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587df015-3244-4e73-b5c4-cec335ac9110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.345174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df015-3244-4e73-b5c4-cec335ac9110 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.345479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df015-3244-4e73-b5c4-cec335ac9110 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.345757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587df015-3244-4e73-b5c4-cec335ac9110 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.346505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587df015-3244-4e73-b5c4-cec335ac9110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.348130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587df015-3244-4e73-b5c4-cec335ac9110 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.358508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2aa384b631244bf87de25b2ff0b8944, user_id=1eb0bee5b68c447dba83ae8742cf5eb6, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.361220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.396565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1033/2059] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:32 2022] PUT /identity/v3/projects/e2aa384b631244bf87de25b2ff0b8944/users/1eb0bee5b68c447dba83ae8742cf5eb6/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:32.396765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1027/2060] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:32.396852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.399939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f324e0a-cfea-42f8-ab38-d9358a656b0f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.400283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f324e0a-cfea-42f8-ab38-d9358a656b0f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.400631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f324e0a-cfea-42f8-ab38-d9358a656b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.406612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:32.458993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7f324e0a-cfea-42f8-ab38-d9358a656b0f None None] MFA Rules not processed for user `1eb0bee5b68c447dba83ae8742cf5eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:32.461144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7f324e0a-cfea-42f8-ab38-d9358a656b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:32.510833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a10bdc-78ef-411a-9ec3-e116ab112b81 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.525940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65b458-b339-4b0e-8e83-1b4515091f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.526827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65b458-b339-4b0e-8e83-1b4515091f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.527105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65b458-b339-4b0e-8e83-1b4515091f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.527379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65b458-b339-4b0e-8e83-1b4515091f0c admin admin] PATH_INFO: `/v3/projects/7daf3318eab14cb89c3c086f2c72c52e/users/0aea084718bb43208024bb770eefe781/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.544834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65b458-b339-4b0e-8e83-1b4515091f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7daf3318eab14cb89c3c086f2c72c52e, user_id=0aea084718bb43208024bb770eefe781, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.547674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65b458-b339-4b0e-8e83-1b4515091f0c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.568017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1034/2061] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:32 2022] PUT /identity/v3/projects/7daf3318eab14cb89c3c086f2c72c52e/users/0aea084718bb43208024bb770eefe781/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:32.578365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a4ec0-1b62-408e-adca-c4810769fb77 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.578683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a4ec0-1b62-408e-adca-c4810769fb77 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.578933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a4ec0-1b62-408e-adca-c4810769fb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.579618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1028/2062] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:32.580980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f324e0a-cfea-42f8-ab38-d9358a656b0f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.591336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:32.597636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f324e0a-cfea-42f8-ab38-d9358a656b0f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.610802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1f6a4ec0-1b62-408e-adca-c4810769fb77 None None] MFA Rules not processed for user `0aea084718bb43208024bb770eefe781`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:32.612575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1f6a4ec0-1b62-408e-adca-c4810769fb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:32.698473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.699270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.699641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.700028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.701029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.703419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.744558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1035/2063] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:32.749951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1029/2064] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:32 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:32.750740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a4ec0-1b62-408e-adca-c4810769fb77 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.756013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.770168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a4ec0-1b62-408e-adca-c4810769fb77 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.837589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.838268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.838552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.839460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.839652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.841815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.842852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460559b3-98b9-466e-9ebe-15a643cf151f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.848670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.849533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.849815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.850524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.865744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1036/2065] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:32 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:32.869750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.885090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc8a00-f61d-40d5-afaa-b386910776b8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.888603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:32.892743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.894168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.894452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.894725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.895475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.897282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.909011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1037/2066] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:32.912977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.924897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924cdb2-1145-4058-8419-43592d2fce10 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.926980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.928837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.934321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba8509-f200-47fa-b416-3a886a91024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.935046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba8509-f200-47fa-b416-3a886a91024d admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.935308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba8509-f200-47fa-b416-3a886a91024d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.935640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba8509-f200-47fa-b416-3a886a91024d admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:32.936399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba8509-f200-47fa-b416-3a886a91024d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:32.938142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba8509-f200-47fa-b416-3a886a91024d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:32.945007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1030/2067] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:32.953121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1038/2068] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:32.957796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:32.970452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48281eb2-afd2-429d-900c-08856b1df8ab service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:32.976840 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:32.977819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:32.978131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:32.978397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] PATH_INFO: `/v3/projects/eec4822ec3314563812c86da509d2e5e/users/7fc6777b9d3b4e409c579b9d00c10948/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.030015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec4822ec3314563812c86da509d2e5e, user_id=7fc6777b9d3b4e409c579b9d00c10948, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:33.032898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:33.061276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1031/2069] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:32 2022] PUT /identity/v3/projects/eec4822ec3314563812c86da509d2e5e/users/7fc6777b9d3b4e409c579b9d00c10948/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:33.065590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b7139-cb12-498b-8abc-eaa543683b32 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.065590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b7139-cb12-498b-8abc-eaa543683b32 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.065590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b7139-cb12-498b-8abc-eaa543683b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.115008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d15b7139-cb12-498b-8abc-eaa543683b32 None None] MFA Rules not processed for user `7fc6777b9d3b4e409c579b9d00c10948`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:33.116364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d15b7139-cb12-498b-8abc-eaa543683b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:33.199522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:33.229288 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1039/2070] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:33.233598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:33.348251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67312d6-e461-4a4f-abec-9a03db8bdf96 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:33.354701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cd862-a47e-4002-a72a-ada8d62de028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:33.355728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cd862-a47e-4002-a72a-ada8d62de028 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.356368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cd862-a47e-4002-a72a-ada8d62de028 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.357234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cd862-a47e-4002-a72a-ada8d62de028 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.358156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cd862-a47e-4002-a72a-ada8d62de028 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:33.360941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0cd862-a47e-4002-a72a-ada8d62de028 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:33.401531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1032/2071] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:33 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:33.405734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b7139-cb12-498b-8abc-eaa543683b32 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:33.423577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b7139-cb12-498b-8abc-eaa543683b32 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:33.565081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:33.566448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.567087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.567802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.569309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:33.572930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:33.598485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1b214-9548-4d18-80e1-0b4d3bc49594 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.598772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1b214-9548-4d18-80e1-0b4d3bc49594 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.599046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1b214-9548-4d18-80e1-0b4d3bc49594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.606614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1040/2072] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:33 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:33.614949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:33.617774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-15b1b214-9548-4d18-80e1-0b4d3bc49594 None None] MFA Rules not processed for user `17123d3e5fe945bbb1cb835f154965d4`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:33.619001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-15b1b214-9548-4d18-80e1-0b4d3bc49594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:33.626807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3aaf4-9ff4-48af-bf05-4c83ac95440a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:33.632434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:33.633256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.633544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.634020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.636749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:33.660804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:33.679178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1041/2073] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:33.684239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:33.714876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1033/2074] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:33.719423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec23592-ff85-4de9-a161-857624160a48 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:33.724913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:33.725820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.726099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.726369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] PATH_INFO: `/v3/projects/778f57f5bd04485b92f2dec5ddab6b3d/users/333bb7d58dd441cc9952efeab9ffc4d9/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.756187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] RBAC: Authorizing `identity:create_grant(project_id=778f57f5bd04485b92f2dec5ddab6b3d, user_id=333bb7d58dd441cc9952efeab9ffc4d9, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:33.760141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:33.793032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1042/2075] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:33 2022] PUT /identity/v3/projects/778f57f5bd04485b92f2dec5ddab6b3d/users/333bb7d58dd441cc9952efeab9ffc4d9/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:33.797269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1b214-9548-4d18-80e1-0b4d3bc49594 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:33.829371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1b214-9548-4d18-80e1-0b4d3bc49594 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:33.906288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:33.950973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b44e1-7fd6-44e0-be70-9eb0055fd4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:33.951660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b44e1-7fd6-44e0-be70-9eb0055fd4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.951941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b44e1-7fd6-44e0-be70-9eb0055fd4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.952242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b44e1-7fd6-44e0-be70-9eb0055fd4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.952975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b44e1-7fd6-44e0-be70-9eb0055fd4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:33.954648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b44e1-7fd6-44e0-be70-9eb0055fd4d6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:33.967320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1034/2076] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:33.971873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:33.990243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7a854-7086-4da4-98ea-32718d06cfde admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:33.996319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:33.997322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:33.997599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:33.997985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] PATH_INFO: `/v3/projects/778f57f5bd04485b92f2dec5ddab6b3d/users/333bb7d58dd441cc9952efeab9ffc4d9/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:33.999889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=778f57f5bd04485b92f2dec5ddab6b3d, user_id=333bb7d58dd441cc9952efeab9ffc4d9, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:34.002984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:34.016817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1043/2077] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:33 2022] PUT /identity/v3/projects/778f57f5bd04485b92f2dec5ddab6b3d/users/333bb7d58dd441cc9952efeab9ffc4d9/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:34.029441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f44c-3fe8-472d-927a-b107823185a2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:34.029726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f44c-3fe8-472d-927a-b107823185a2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:34.030000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445f44c-3fe8-472d-927a-b107823185a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:34.057561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c445f44c-3fe8-472d-927a-b107823185a2 None None] MFA Rules not processed for user `333bb7d58dd441cc9952efeab9ffc4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:34.058792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c445f44c-3fe8-472d-927a-b107823185a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:34.141522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1035/2078] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:34.153951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:34.218599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a4e1e-7dc4-458c-8604-48faffcd8a17 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:34.222300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:34.222956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:34.223227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:34.223508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:34.234641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:34.236579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:34.257760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1044/2079] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:34.311423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f44c-3fe8-472d-927a-b107823185a2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:34.323020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445f44c-3fe8-472d-927a-b107823185a2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:34.327310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4732d66-c3f2-43b9-af6f-aa131a0080bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:34.328055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732d66-c3f2-43b9-af6f-aa131a0080bf service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:34.328439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732d66-c3f2-43b9-af6f-aa131a0080bf service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:34.328812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732d66-c3f2-43b9-af6f-aa131a0080bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:34.402982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4732d66-c3f2-43b9-af6f-aa131a0080bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:34.404600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4732d66-c3f2-43b9-af6f-aa131a0080bf service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:34.414488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1036/2080] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:36.269436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:36.271274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:36.336668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:36.369825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd9442-cb55-4306-86fd-f4379ff29d67 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:36.374781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:36.375748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:36.376118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:36.376453 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] PATH_INFO: `/v3/users/c46a0a0e45694029934464c2cda9d5c9` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:36.377858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c46a0a0e45694029934464c2cda9d5c9)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:36.379906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:36.447368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c46a0a0e45694029934464c2cda9d5c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.461968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c46a0a0e45694029934464c2cda9d5c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.473374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c46a0a0e45694029934464c2cda9d5c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.474224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c46a0a0e45694029934464c2cda9d5c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.475148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c46a0a0e45694029934464c2cda9d5c9'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.502755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1045/2081] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:36 2022] DELETE /identity/v3/users/c46a0a0e45694029934464c2cda9d5c9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:36.507378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:36.507875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:36.507964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:36.565753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:36.569994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:36.659433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:36.670220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1037/2082] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:36.680474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:36.681929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:36.741267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:36.743871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:36.757841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9c3ea-35e5-48a0-b673-6071bc421e08 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:36.763045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969dec5-1135-4af9-ad2d-b45aa872e284 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:36.765604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:36.766691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:36.766971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:36.767290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] PATH_INFO: `/v3/users/0aea084718bb43208024bb770eefe781` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:36.768347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db38b47-0ad6-43e1-b9a0-14da786daa7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:36.768524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aea084718bb43208024bb770eefe781)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:36.769183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db38b47-0ad6-43e1-b9a0-14da786daa7c service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:36.769749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db38b47-0ad6-43e1-b9a0-14da786daa7c service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:36.770030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db38b47-0ad6-43e1-b9a0-14da786daa7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:36.770650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:36.781309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db38b47-0ad6-43e1-b9a0-14da786daa7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:36.782850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db38b47-0ad6-43e1-b9a0-14da786daa7c service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:36.794721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1046/2083] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:36.843175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aea084718bb43208024bb770eefe781'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.843175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aea084718bb43208024bb770eefe781'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.853344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aea084718bb43208024bb770eefe781'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.863564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aea084718bb43208024bb770eefe781'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.864870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aea084718bb43208024bb770eefe781'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:36.880843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1038/2084] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:36 2022] DELETE /identity/v3/users/0aea084718bb43208024bb770eefe781 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:36.886439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:36.886723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:36.887050 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:36.910399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:36.912101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.004071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:37.015605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1047/2085] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:37.031860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.033527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.100740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a73b7-6f7e-4be6-a16a-01cb5f26e2a8 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:37.104663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:37.105525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.105897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.106230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.120736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:37.122841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:37.136275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1039/2086] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:37.236836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.238377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.263060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7cc09-c713-4ef8-afce-6050d0a003f7 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:37.272142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:37.272988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.273429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.273699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] PATH_INFO: `/v3/projects/24bfd2ba81104fe48a55f57942474bcd` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.281913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24bfd2ba81104fe48a55f57942474bcd)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:37.285225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:37.360819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24bfd2ba81104fe48a55f57942474bcd', 'request_id': 'req-1dee0640-ea63-4587-9658-5db28be0ad94'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.368097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24bfd2ba81104fe48a55f57942474bcd', 'request_id': 'req-1dee0640-ea63-4587-9658-5db28be0ad94'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.382419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24bfd2ba81104fe48a55f57942474bcd', 'request_id': 'req-1dee0640-ea63-4587-9658-5db28be0ad94'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.398181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] The token cache is being invalidate because project 24bfd2ba81104fe48a55f57942474bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:37.398501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.399416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.401051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1048/2087] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:37 2022] DELETE /identity/v3/projects/24bfd2ba81104fe48a55f57942474bcd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:37.405027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.416313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c2f4a-4d25-4370-83a1-ccb4f1193c01 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:37.510467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:37.511391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.511657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.512198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] PATH_INFO: `/v3/users/54eeca147da941168d270ecc07e17b03` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.513464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] RBAC: Authorizing `identity:delete_user(user_id=54eeca147da941168d270ecc07e17b03)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:37.515442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:37.531452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.532426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:37.533228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.551150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee0640-ea63-4587-9658-5db28be0ad94 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:37.557006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:37.558899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.559350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.560075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] PATH_INFO: `/v3/projects/7daf3318eab14cb89c3c086f2c72c52e` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.561797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7daf3318eab14cb89c3c086f2c72c52e)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:37.563954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:37.600920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54eeca147da941168d270ecc07e17b03'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.602287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54eeca147da941168d270ecc07e17b03'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.621151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54eeca147da941168d270ecc07e17b03'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.624967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7daf3318eab14cb89c3c086f2c72c52e', 'request_id': 'req-a704270c-ece9-48b6-a0e9-7e034c34051e'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.632081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54eeca147da941168d270ecc07e17b03'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.632891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9c6a9f3c-08a5-49c4-ba37-051a3714dfed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54eeca147da941168d270ecc07e17b03'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.635122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7daf3318eab14cb89c3c086f2c72c52e', 'request_id': 'req-a704270c-ece9-48b6-a0e9-7e034c34051e'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.646899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7daf3318eab14cb89c3c086f2c72c52e', 'request_id': 'req-a704270c-ece9-48b6-a0e9-7e034c34051e'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.652421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1040/2088] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:37 2022] DELETE /identity/v3/users/54eeca147da941168d270ecc07e17b03 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:37.663395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] The token cache is being invalidate because project 7daf3318eab14cb89c3c086f2c72c52e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:37.663803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.664722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.666627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1049/2089] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:37 2022] DELETE /identity/v3/projects/7daf3318eab14cb89c3c086f2c72c52e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:37.672226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a05e4-5b37-4259-85ad-6bc4c210bca5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.672538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a05e4-5b37-4259-85ad-6bc4c210bca5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.672819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a05e4-5b37-4259-85ad-6bc4c210bca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.707624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.709987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.712349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e50a05e4-5b37-4259-85ad-6bc4c210bca5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:37.713868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e50a05e4-5b37-4259-85ad-6bc4c210bca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.794248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704270c-ece9-48b6-a0e9-7e034c34051e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:37.798866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1041/2090] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:37.806557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e40d07-f777-45ad-87ef-10bdfd73c7c6 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.807090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e40d07-f777-45ad-87ef-10bdfd73c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.807472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:37.807928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e40d07-f777-45ad-87ef-10bdfd73c7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.809884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.810239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.810594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] PATH_INFO: `/v3/users/1eb0bee5b68c447dba83ae8742cf5eb6` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.812049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb0bee5b68c447dba83ae8742cf5eb6)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:37.815088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:37.829789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:37.845052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f4e40d07-f777-45ad-87ef-10bdfd73c7c6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:37.846782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f4e40d07-f777-45ad-87ef-10bdfd73c7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.862896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1042/2091] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:37.863795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e40d07-f777-45ad-87ef-10bdfd73c7c6 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.875977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e40d07-f777-45ad-87ef-10bdfd73c7c6 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:37.880973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:37.881805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.882095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.882484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.883449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:37.883774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:37.885428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:37.905811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1043/2092] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:37.906747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.912128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb0bee5b68c447dba83ae8742cf5eb6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.919297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d2023-97ca-4334-8a0f-02c977256f91 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:37.924442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:37.925644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.925644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.925838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:37.926496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:37.927272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb0bee5b68c447dba83ae8742cf5eb6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.928632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:37.940226 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb0bee5b68c447dba83ae8742cf5eb6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.941584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb0bee5b68c447dba83ae8742cf5eb6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.942359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bbcce779-6bf7-4a10-8240-f16aab153858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb0bee5b68c447dba83ae8742cf5eb6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:37.965166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1044/2093] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:37 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:37.968583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:37.972591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:37.975564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1050/2094] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:37 2022] DELETE /identity/v3/users/1eb0bee5b68c447dba83ae8742cf5eb6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:37.980558 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:37.981036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:37.981503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.016530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:38.018197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.073349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:38.111592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfcba-a815-4d5c-8362-348a938223a9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.116775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1051/2095] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:38.116775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.117404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.126442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.127829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.128262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.129585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.130703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.133898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8e247-fe4f-402a-b700-6486d11debe7 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.134961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.139522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.140662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.140917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.141209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] PATH_INFO: `/v3/projects/e972d2524081496b9ac83a85e4b4c751` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.142601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e972d2524081496b9ac83a85e4b4c751)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.144477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.170084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1045/2096] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:37 2022] POST /identity/v3/users => generated 288 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:38.171923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.174182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.206161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e972d2524081496b9ac83a85e4b4c751', 'request_id': 'req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:38.212931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e972d2524081496b9ac83a85e4b4c751', 'request_id': 'req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:38.223135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e972d2524081496b9ac83a85e4b4c751', 'request_id': 'req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:38.236427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] The token cache is being invalidate because project e972d2524081496b9ac83a85e4b4c751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:38.236737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:38.237419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:38.239805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1052/2097] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:38 2022] DELETE /identity/v3/projects/e972d2524081496b9ac83a85e4b4c751 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:38.240412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.245488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.247349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e114bd-0b1b-4bbc-ac42-00cf25384e14 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.258314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:38.311304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba24f5-59f6-4737-a567-bf52917e8e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.312098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba24f5-59f6-4737-a567-bf52917e8e34 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.312463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba24f5-59f6-4737-a567-bf52917e8e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.312817 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba24f5-59f6-4737-a567-bf52917e8e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.314870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0dba24f5-59f6-4737-a567-bf52917e8e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.333262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b068e8-f25e-4c1f-aa8c-5eca4afa578f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.339119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b02baf-b328-49a9-80a2-950164a989d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.339796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02baf-b328-49a9-80a2-950164a989d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.340162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02baf-b328-49a9-80a2-950164a989d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.340442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b02baf-b328-49a9-80a2-950164a989d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.341180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b02baf-b328-49a9-80a2-950164a989d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.343246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b02baf-b328-49a9-80a2-950164a989d8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.357099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1053/2098] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:38.357099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2349cc-0d4e-4a30-bad7-1390e7de0a1b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.357099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2349cc-0d4e-4a30-bad7-1390e7de0a1b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.357099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2349cc-0d4e-4a30-bad7-1390e7de0a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.376329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-bd2349cc-0d4e-4a30-bad7-1390e7de0a1b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:38.377552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bd2349cc-0d4e-4a30-bad7-1390e7de0a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.386233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba24f5-59f6-4737-a567-bf52917e8e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.393942 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1054/2099] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:38.394166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba24f5-59f6-4737-a567-bf52917e8e34 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.394268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2349cc-0d4e-4a30-bad7-1390e7de0a1b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.400076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2349cc-0d4e-4a30-bad7-1390e7de0a1b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.401677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1046/2100] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:38.406285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.407042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.407393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.407737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.408746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.409935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11aad7-97ac-4d27-9738-ea0bf2906b80 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.410387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11aad7-97ac-4d27-9738-ea0bf2906b80 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.410914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11aad7-97ac-4d27-9738-ea0bf2906b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.411039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.424897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1055/2101] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:38.429037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.439430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4d11aad7-97ac-4d27-9738-ea0bf2906b80 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:38.443746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4d11aad7-97ac-4d27-9738-ea0bf2906b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.446084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e27bc2-80ff-47af-9d2d-552581bc244d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.458768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a7065-7259-4380-afaa-478eeca2d0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.459566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7065-7259-4380-afaa-478eeca2d0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.460677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7065-7259-4380-afaa-478eeca2d0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.461034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a7065-7259-4380-afaa-478eeca2d0b2 admin admin] PATH_INFO: `/v3/projects/272515157873449cbe8ed5b774421c96/users/c83c0955b7794d7393d09d5b04185897/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.466850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1047/2102] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:38.466850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11aad7-97ac-4d27-9738-ea0bf2906b80 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.487957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11aad7-97ac-4d27-9738-ea0bf2906b80 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.491042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7065-7259-4380-afaa-478eeca2d0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=272515157873449cbe8ed5b774421c96, user_id=c83c0955b7794d7393d09d5b04185897, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.495969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a7065-7259-4380-afaa-478eeca2d0b2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.500373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.501223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.501580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.501979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.504497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:38.504805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.507183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.524255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1056/2103] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:38 2022] PUT /identity/v3/projects/272515157873449cbe8ed5b774421c96/users/c83c0955b7794d7393d09d5b04185897/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:38.534771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1048/2104] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:38.535894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027eb71-9744-4cca-87e8-1a5b87829c8a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.536590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027eb71-9744-4cca-87e8-1a5b87829c8a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.536876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027eb71-9744-4cca-87e8-1a5b87829c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.539149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.561544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d0ca8-bb44-4156-bfed-df0394d62cc0 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.572119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-c027eb71-9744-4cca-87e8-1a5b87829c8a None None] MFA Rules not processed for user `c83c0955b7794d7393d09d5b04185897`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:38.574107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c027eb71-9744-4cca-87e8-1a5b87829c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.604747 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:38.664248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.665681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.666134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.666568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.667490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.670184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.674137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1057/2105] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:38.675300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027eb71-9744-4cca-87e8-1a5b87829c8a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.689181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027eb71-9744-4cca-87e8-1a5b87829c8a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.695749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.696495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.697058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.697339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.698042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.704389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.713011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1049/2106] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:38 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:38.713926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.763680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1058/2107] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:38 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:38.768886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.814113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:38.827371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcfe029-1e3a-4556-9818-f22b7e22c58d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.850418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4c46-c72f-4281-a8c9-205c8ebce325 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.855815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.856542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.856823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.857441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.857927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.859732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.885485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1059/2108] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:38 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:38.885485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.885485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:38.897508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f10b96-5456-4b77-987f-c320caa3a87d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.902322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.903603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.903906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.904230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] PATH_INFO: `/v3/projects/e2aa384b631244bf87de25b2ff0b8944` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.905422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2aa384b631244bf87de25b2ff0b8944)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.906138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.906821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.907097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.907363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.907625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.908344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.910290 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.940565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1050/2109] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:38 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:38.942107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:38.960511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dfefcd-393a-49f5-a13b-848a64c7ebcb admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:38.966620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:38.967556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:38.967837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:38.968420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:38.969171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:38.971130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:38.990294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1051/2110] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:38.990294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.002061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066f6df-8842-4ddd-9ebb-0e2aa5329ce1 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.007665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2aa384b631244bf87de25b2ff0b8944', 'request_id': 'req-12546852-f1c0-465a-bfdf-d43fa9fd338a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.015588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2aa384b631244bf87de25b2ff0b8944', 'request_id': 'req-12546852-f1c0-465a-bfdf-d43fa9fd338a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.025422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2aa384b631244bf87de25b2ff0b8944', 'request_id': 'req-12546852-f1c0-465a-bfdf-d43fa9fd338a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.041565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] The token cache is being invalidate because project e2aa384b631244bf87de25b2ff0b8944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:39.042231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.043329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.045437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1060/2111] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:38 2022] DELETE /identity/v3/projects/e2aa384b631244bf87de25b2ff0b8944 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:39.046263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:39.046508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.051293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.081893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.082591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.082885 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.083149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.086382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.088309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.110706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1052/2112] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:39.110706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.110706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.127645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3f254-6cbc-41d9-9fcc-9150aca558cc admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.128568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12546852-f1c0-465a-bfdf-d43fa9fd338a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.133062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.133257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.133903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.134857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.134978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.135070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.135144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.135220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] PATH_INFO: `/v3/users/7fc6777b9d3b4e409c579b9d00c10948` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.135317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.136247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc6777b9d3b4e409c579b9d00c10948)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.136801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.138355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.148771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1061/2113] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:39.149910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.153639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.166663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14592add-456b-4544-91d7-6d7290501673 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.174522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.175753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.176105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.176523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.177270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.179059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.193260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1062/2114] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:39.193260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.198197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc6777b9d3b4e409c579b9d00c10948'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.199200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc6777b9d3b4e409c579b9d00c10948'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.204645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f9dcb-cb1f-45ca-9f0d-8cba84ce410a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.209853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.210667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.210922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.211202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] PATH_INFO: `/v3/projects/8fc7a9e82ea94ce7b0cefcf840f723b0/users/2201396db9c14467b0fa24dd72b02c75/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.222061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc6777b9d3b4e409c579b9d00c10948'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.228055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc7a9e82ea94ce7b0cefcf840f723b0, user_id=2201396db9c14467b0fa24dd72b02c75, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.231600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.237038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc6777b9d3b4e409c579b9d00c10948'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.238444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-46f7f753-0e12-4531-a6be-fff48326ec2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc6777b9d3b4e409c579b9d00c10948'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.253006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1063/2115] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:39 2022] PUT /identity/v3/projects/8fc7a9e82ea94ce7b0cefcf840f723b0/users/2201396db9c14467b0fa24dd72b02c75/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:39.253765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.255323 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.263825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:39.277904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1053/2116] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:39 2022] DELETE /identity/v3/users/7fc6777b9d3b4e409c579b9d00c10948 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:39.287777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.288238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.288525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.319189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] MFA Rules not processed for user `2201396db9c14467b0fa24dd72b02c75`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:39.320858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.367187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271333c-352c-4293-a0ba-d547724da4f7 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.444994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1054/2117] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:39.450787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.454697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.467368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7913532-94e8-42b7-8e09-6989e344ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.470410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913532-94e8-42b7-8e09-6989e344ec45 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.471372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913532-94e8-42b7-8e09-6989e344ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.471845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7913532-94e8-42b7-8e09-6989e344ec45 admin admin] PATH_INFO: `/v3/projects/10b990353257486e9c2539f748bf3431/users/19ee0bee9bfc4803aeb4216a598b03dc/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.497514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913532-94e8-42b7-8e09-6989e344ec45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10b990353257486e9c2539f748bf3431, user_id=19ee0bee9bfc4803aeb4216a598b03dc, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.501742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7913532-94e8-42b7-8e09-6989e344ec45 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.510045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:39.526805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1064/2118] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:39 2022] PUT /identity/v3/projects/10b990353257486e9c2539f748bf3431/users/19ee0bee9bfc4803aeb4216a598b03dc/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:39.540225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696357a-ddc3-42a7-b12a-3f1cb0eb5cb4 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.540697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696357a-ddc3-42a7-b12a-3f1cb0eb5cb4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.540807 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696357a-ddc3-42a7-b12a-3f1cb0eb5cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.563214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103399d9-0fee-497d-8126-f9a91349d8b9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.575797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-5696357a-ddc3-42a7-b12a-3f1cb0eb5cb4 None None] MFA Rules not processed for user `19ee0bee9bfc4803aeb4216a598b03dc`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:39.577092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5696357a-ddc3-42a7-b12a-3f1cb0eb5cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.655424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.656731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.657570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.657952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.658938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.659413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1065/2119] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:39.660330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696357a-ddc3-42a7-b12a-3f1cb0eb5cb4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.665635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.673616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696357a-ddc3-42a7-b12a-3f1cb0eb5cb4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.681998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.682758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.683104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.683448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.684303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.686666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.713086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1055/2120] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:39 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:39.714232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.716392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.728469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1066/2121] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:39 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:39.729460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.736957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:39.812662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e92407-1d29-4e0e-a35d-eff6cb263c82 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.813306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f98381-7570-406f-a89b-b23c7a94162f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.818781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.819441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.819880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.820300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.820690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.820972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.821271 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.821374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] PATH_INFO: `/v3/projects/eec4822ec3314563812c86da509d2e5e` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.822158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.823089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec4822ec3314563812c86da509d2e5e)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.825051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.825252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.854777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1067/2122] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:39 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:39.856190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.870277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eadc93-10f4-4009-8743-4ef8363c2006 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:39.884228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:39.885263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:39.885989 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:39.886370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:39.887569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:39.892264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec4822ec3314563812c86da509d2e5e', 'request_id': 'req-193eac61-c494-470b-a13b-4404beaa120e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.892628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:39.907552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec4822ec3314563812c86da509d2e5e', 'request_id': 'req-193eac61-c494-470b-a13b-4404beaa120e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.926086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec4822ec3314563812c86da509d2e5e', 'request_id': 'req-193eac61-c494-470b-a13b-4404beaa120e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.926897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1068/2123] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:39 2022] POST /identity/v3/users => generated 293 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:39.929391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.949322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:39.956394 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] The token cache is being invalidate because project eec4822ec3314563812c86da509d2e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:39.956748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.957419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:39.959194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1056/2124] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:39 2022] DELETE /identity/v3/projects/eec4822ec3314563812c86da509d2e5e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:39.964890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:39.966627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:39.985143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f9efc-4962-4681-893a-23d4abf1e5d1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.069316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193eac61-c494-470b-a13b-4404beaa120e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.075854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.076589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.076873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.077317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.078285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.080322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.096232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1057/2125] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:40.097106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.100052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.105813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.106824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.107312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.107791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.108755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.111827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.117929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d024c-3c83-4a10-87fd-9b874fde5ae7 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.124408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.125785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.126359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.127637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] PATH_INFO: `/v3/users/333bb7d58dd441cc9952efeab9ffc4d9` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.128097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333bb7d58dd441cc9952efeab9ffc4d9)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.130485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.154239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1069/2126] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:40.155358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.177606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f245d3f-73f0-44e6-a3f4-e56f940e839d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.183358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.184455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.184925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.185266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] PATH_INFO: `/v3/projects/4e19801764754ac8bcc23c5a8313ccec/users/19035d063cf14b46a975b4645a2a2828/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.206194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e19801764754ac8bcc23c5a8313ccec, user_id=19035d063cf14b46a975b4645a2a2828, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.209815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.232725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1070/2127] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:40 2022] PUT /identity/v3/projects/4e19801764754ac8bcc23c5a8313ccec/users/19035d063cf14b46a975b4645a2a2828/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:40.234261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.237073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.240110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333bb7d58dd441cc9952efeab9ffc4d9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.241028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333bb7d58dd441cc9952efeab9ffc4d9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.258939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333bb7d58dd441cc9952efeab9ffc4d9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.286897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333bb7d58dd441cc9952efeab9ffc4d9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.287568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333bb7d58dd441cc9952efeab9ffc4d9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.307036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1058/2128] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:40 2022] DELETE /identity/v3/users/333bb7d58dd441cc9952efeab9ffc4d9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:40.307902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.309348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.313212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:40.336784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1989d7d-9c8a-4e63-bb28-1272a5d36ba1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.426870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.427927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.428641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.428926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] PATH_INFO: `/v3/projects/33cb711c87da4009969876ca5455041b/users/4765b09fb948458ca4200ab1815d8e44/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.431477 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7c254-9871-4581-a75c-9ed448c9fa12 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.437932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.439041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.439471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.440136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.441029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.443410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.452433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cb711c87da4009969876ca5455041b, user_id=4765b09fb948458ca4200ab1815d8e44, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.456108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.471400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1059/2129] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:40.473271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.478640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1071/2130] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:40 2022] PUT /identity/v3/projects/33cb711c87da4009969876ca5455041b/users/4765b09fb948458ca4200ab1815d8e44/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:40.479781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:40.483531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.486373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.490081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ab08f-a535-41e2-8bf9-72dd72e2fe92 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.599668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532c57b-6955-487c-a149-ce255f3a668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.600784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532c57b-6955-487c-a149-ce255f3a668c admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.601089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532c57b-6955-487c-a149-ce255f3a668c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.601351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532c57b-6955-487c-a149-ce255f3a668c admin admin] PATH_INFO: `/v3/projects/4e19801764754ac8bcc23c5a8313ccec/users/19035d063cf14b46a975b4645a2a2828/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.605719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532c57b-6955-487c-a149-ce255f3a668c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e19801764754ac8bcc23c5a8313ccec, user_id=19035d063cf14b46a975b4645a2a2828, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.610016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532c57b-6955-487c-a149-ce255f3a668c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.632586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1060/2131] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:40 2022] PUT /identity/v3/projects/4e19801764754ac8bcc23c5a8313ccec/users/19035d063cf14b46a975b4645a2a2828/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:40.635276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0676a86-2f45-4eff-823d-842b885cd60c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.635571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0676a86-2f45-4eff-823d-842b885cd60c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.635850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0676a86-2f45-4eff-823d-842b885cd60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.636553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188acd56-beeb-4ea2-be63-ad3ac6eebe35 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.683555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a0676a86-2f45-4eff-823d-842b885cd60c None None] MFA Rules not processed for user `19035d063cf14b46a975b4645a2a2828`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:40.685201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a0676a86-2f45-4eff-823d-842b885cd60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.750535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.751267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.751609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.751943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.752755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.754442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.767723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1072/2132] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:40.768700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.772795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.789985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911aeda-6349-471a-8522-d128109cbd93 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.790103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1061/2133] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:40.790103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0676a86-2f45-4eff-823d-842b885cd60c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.793219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.797065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.797065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.797065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] PATH_INFO: `/v3/projects/778f57f5bd04485b92f2dec5ddab6b3d` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.798629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778f57f5bd04485b92f2dec5ddab6b3d)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.801421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.803690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0676a86-2f45-4eff-823d-842b885cd60c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.808508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.809585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.810383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.810383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] PATH_INFO: `/v3/projects/33cb711c87da4009969876ca5455041b/users/4765b09fb948458ca4200ab1815d8e44/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.812428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cb711c87da4009969876ca5455041b, user_id=4765b09fb948458ca4200ab1815d8e44, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:40.815115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:40.820078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:40.826677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1062/2134] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:40 2022] PUT /identity/v3/projects/33cb711c87da4009969876ca5455041b/users/4765b09fb948458ca4200ab1815d8e44/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:40.827661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:40.829851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.868854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778f57f5bd04485b92f2dec5ddab6b3d', 'request_id': 'req-97d0b41a-b294-4917-9519-3fd61d510310'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.876186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778f57f5bd04485b92f2dec5ddab6b3d', 'request_id': 'req-97d0b41a-b294-4917-9519-3fd61d510310'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.885129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778f57f5bd04485b92f2dec5ddab6b3d', 'request_id': 'req-97d0b41a-b294-4917-9519-3fd61d510310'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.891624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012a202-6543-4e31-98e5-a03f5e0472aa admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:40.896471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] The token cache is being invalidate because project 778f57f5bd04485b92f2dec5ddab6b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:40.896755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.897408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-97d0b41a-b294-4917-9519-3fd61d510310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:40.903591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1073/2135] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:40 2022] DELETE /identity/v3/projects/778f57f5bd04485b92f2dec5ddab6b3d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:40.903591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530827c-6304-470e-92da-5affa09820f1 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.903591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530827c-6304-470e-92da-5affa09820f1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.903591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530827c-6304-470e-92da-5affa09820f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.922224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:40.932201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-4530827c-6304-470e-92da-5affa09820f1 None None] MFA Rules not processed for user `4765b09fb948458ca4200ab1815d8e44`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:40.934237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-4530827c-6304-470e-92da-5affa09820f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:40.954688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:40.955350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:40.955634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:40.955910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:40.958034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:41.015886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1074/2136] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:41.018236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc105e-b6c6-4e3d-8253-321d4b580f6d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.018525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc105e-b6c6-4e3d-8253-321d4b580f6d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.018962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc105e-b6c6-4e3d-8253-321d4b580f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.037476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.038945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.041312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-10dc105e-b6c6-4e3d-8253-321d4b580f6d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:41.042751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-10dc105e-b6c6-4e3d-8253-321d4b580f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:41.049938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1063/2137] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:41.050895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.052418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:41.064062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d804f40c-4606-45e4-b347-e474f141c471 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.079045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.079915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.080264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.080597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.083086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:41.149356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1075/2138] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:41.151420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69229-51b1-4b23-a257-88b7d2d14211 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.152158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69229-51b1-4b23-a257-88b7d2d14211 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.152621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69229-51b1-4b23-a257-88b7d2d14211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.166881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.168897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.173196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-59b69229-51b1-4b23-a257-88b7d2d14211 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:41.174828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-59b69229-51b1-4b23-a257-88b7d2d14211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:41.180537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1064/2139] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:41.189698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1076/2140] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:41.190661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.203469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c7589-6421-454e-8774-9507ecc7ff85 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.208281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.208953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.209238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.209518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.210392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:41.210776 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.212589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.225383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1065/2141] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:51:41.229728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b69229-51b1-4b23-a257-88b7d2d14211 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.240952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b69229-51b1-4b23-a257-88b7d2d14211 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.249381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.250052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.250336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.250615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.251326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.253490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.285855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:41.297260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1077/2142] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:41 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:41.301275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.312318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927928a-fbc6-49e6-b945-beac0fc47dee admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.380149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.380959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.381306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.381649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.382439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.394849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.406755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:41.434023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.447619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1066/2143] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:41 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:41.454404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e429e2ee-a4f4-43a7-b1e2-006b2f8b97dd admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.462679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.463605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.463912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.464243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.465130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.467720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.488821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.500490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1078/2144] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:41.505130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cc7b2-3c12-4cc9-9fa7-44489c580822 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.513179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65e620-68e7-4573-95ef-f597a011d83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.514256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65e620-68e7-4573-95ef-f597a011d83a admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.514629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65e620-68e7-4573-95ef-f597a011d83a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.514903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65e620-68e7-4573-95ef-f597a011d83a admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.515652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65e620-68e7-4573-95ef-f597a011d83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.518588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65e620-68e7-4573-95ef-f597a011d83a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.541311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1067/2145] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:41.546428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.559009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dadf91-d502-4526-bf5f-ba59ba6b4f50 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.564344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.565654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.566095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.566379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] PATH_INFO: `/v3/projects/12f985b7b3a349508da20bf8ecc49329/users/d1a0ff9363164869ae1c4dc829f2e381/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.590903 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f985b7b3a349508da20bf8ecc49329, user_id=d1a0ff9363164869ae1c4dc829f2e381, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.593754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.626759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1079/2146] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:41 2022] PUT /identity/v3/projects/12f985b7b3a349508da20bf8ecc49329/users/d1a0ff9363164869ae1c4dc829f2e381/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:41.631318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7338-1fe2-4762-a914-888aa1ccdf9f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.631818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7338-1fe2-4762-a914-888aa1ccdf9f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.631915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a7338-1fe2-4762-a914-888aa1ccdf9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.679176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-948a7338-1fe2-4762-a914-888aa1ccdf9f None None] MFA Rules not processed for user `d1a0ff9363164869ae1c4dc829f2e381`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:41.680485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-948a7338-1fe2-4762-a914-888aa1ccdf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:41.749139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1068/2147] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:41.761116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.822583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2cd98-275e-4f63-9b08-ec1c79c04d16 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.826630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a4b9d-2e09-4161-bc85-788271d3dc45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.827328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a4b9d-2e09-4161-bc85-788271d3dc45 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.827622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a4b9d-2e09-4161-bc85-788271d3dc45 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.827880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a4b9d-2e09-4161-bc85-788271d3dc45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.838070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a4b9d-2e09-4161-bc85-788271d3dc45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.839531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a4b9d-2e09-4161-bc85-788271d3dc45 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:41.849731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1080/2148] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:41.870081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7338-1fe2-4762-a914-888aa1ccdf9f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:41.935781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a7338-1fe2-4762-a914-888aa1ccdf9f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:41.941207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:41.942115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:41.942395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:41.942676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] PATH_INFO: `/v3/users/d1a0ff9363164869ae1c4dc829f2e381` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:41.944052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a0ff9363164869ae1c4dc829f2e381)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:41.946070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:42.010354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a0ff9363164869ae1c4dc829f2e381'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.011436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a0ff9363164869ae1c4dc829f2e381'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.036452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a0ff9363164869ae1c4dc829f2e381'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.045201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a0ff9363164869ae1c4dc829f2e381'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.046214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a0ff9363164869ae1c4dc829f2e381'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.048929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:42.065834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1069/2149] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:41 2022] DELETE /identity/v3/users/d1a0ff9363164869ae1c4dc829f2e381 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:42.067373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:42.067761 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:42.067965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:42.089474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:42.090586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:42.152734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1081/2150] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:42.162393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:42.163982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:42.224854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0099276f-33f9-4dfe-88b9-5339613315bf admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:42.228617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e7a4f-c055-4fb5-a6fd-b803f5543b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:42.229269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7a4f-c055-4fb5-a6fd-b803f5543b6e service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:42.229552 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7a4f-c055-4fb5-a6fd-b803f5543b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:42.229828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e7a4f-c055-4fb5-a6fd-b803f5543b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:42.240234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e7a4f-c055-4fb5-a6fd-b803f5543b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:42.241682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e7a4f-c055-4fb5-a6fd-b803f5543b6e service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:42.252306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1070/2151] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:42.586533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:42.589495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:42.607041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87898d0a-3d8e-4f6d-bdaa-1c55a02e07b3 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:42.614233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:42.615732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:42.616045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:42.616331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] PATH_INFO: `/v3/projects/12f985b7b3a349508da20bf8ecc49329` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:42.617561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f985b7b3a349508da20bf8ecc49329)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:42.619605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:42.678235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f985b7b3a349508da20bf8ecc49329', 'request_id': 'req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.689070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f985b7b3a349508da20bf8ecc49329', 'request_id': 'req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.701916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f985b7b3a349508da20bf8ecc49329', 'request_id': 'req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.705594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:42.721389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] The token cache is being invalidate because project 12f985b7b3a349508da20bf8ecc49329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:42.721865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.723186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c9fe2d84-3287-42a5-a02b-5ac6ad70f63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:42.729789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1082/2152] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:42 2022] DELETE /identity/v3/projects/12f985b7b3a349508da20bf8ecc49329 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:42.731368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf363-0c60-4761-88eb-ff2f859e5ddf None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:42.731846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf363-0c60-4761-88eb-ff2f859e5ddf None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:42.732246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bf363-0c60-4761-88eb-ff2f859e5ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:42.753218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-364bf363-0c60-4761-88eb-ff2f859e5ddf None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:42.754570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-364bf363-0c60-4761-88eb-ff2f859e5ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:42.845520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1071/2153] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:42.854995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c3d4c-ea4e-4b52-828b-cadc7f2dc4f4 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:42.855287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c3d4c-ea4e-4b52-828b-cadc7f2dc4f4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:42.855589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c3d4c-ea4e-4b52-828b-cadc7f2dc4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:42.876683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-195c3d4c-ea4e-4b52-828b-cadc7f2dc4f4 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:42.877916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-195c3d4c-ea4e-4b52-828b-cadc7f2dc4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:42.896054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1083/2154] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:42.898098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf363-0c60-4761-88eb-ff2f859e5ddf None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:42.915523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bf363-0c60-4761-88eb-ff2f859e5ddf None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:42.920751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:42.921620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:42.921904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:42.922179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:42.922929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:42.923288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:42.924984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:42.942873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1072/2155] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:42.944513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c3d4c-ea4e-4b52-828b-cadc7f2dc4f4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:42.970767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c3d4c-ea4e-4b52-828b-cadc7f2dc4f4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:42.979133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:42.979648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:42.980665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:42.980665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:42.981886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:42.985523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.015904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.017738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:43.031875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1084/2156] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:42 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:43.036173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.082992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6456c0b6-5229-41e5-9129-48d108b6b28f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.102097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8217c1-7ba0-4dbb-ac8b-e5098759758a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.108675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.109337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.109657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.110257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.110601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.112735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.122410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:43.134029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1085/2157] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:43 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:43.138321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.139529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a938977-0cbb-48ca-b7f5-5ddc3be43fdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.140216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938977-0cbb-48ca-b7f5-5ddc3be43fdc service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.140496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938977-0cbb-48ca-b7f5-5ddc3be43fdc service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.140769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938977-0cbb-48ca-b7f5-5ddc3be43fdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.142687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4a938977-0cbb-48ca-b7f5-5ddc3be43fdc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:43.148366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029df2c4-d8a5-4530-890d-2de4c61432d8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.152945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.153589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.153866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.154132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.154849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.156464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.166504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1086/2158] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:43.170433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.180806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327f94e-ded7-42fa-8140-4cd1be113823 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.186199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.186839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.187116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.187383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.188216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.189755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.200949 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1087/2159] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:43.203896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a938977-0cbb-48ca-b7f5-5ddc3be43fdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.205426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a938977-0cbb-48ca-b7f5-5ddc3be43fdc service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.208356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.214506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1073/2160] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:43.218008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14381804-be57-47b5-8ac4-f892be8d6b3c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.222466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.223328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.223604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.223872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] PATH_INFO: `/v3/projects/c5e79a83ac1f481f8544b6b1031fe006/users/7378d0a9a8b84a5783b0f8267de64e63/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.238721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e79a83ac1f481f8544b6b1031fe006, user_id=7378d0a9a8b84a5783b0f8267de64e63, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.241780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.269425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1088/2161] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:43 2022] PUT /identity/v3/projects/c5e79a83ac1f481f8544b6b1031fe006/users/7378d0a9a8b84a5783b0f8267de64e63/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:43.275322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee916cd-c966-4ab2-b355-73409acccecb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.275617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee916cd-c966-4ab2-b355-73409acccecb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.275901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee916cd-c966-4ab2-b355-73409acccecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.309524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-dee916cd-c966-4ab2-b355-73409acccecb None None] MFA Rules not processed for user `7378d0a9a8b84a5783b0f8267de64e63`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:43.310228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-dee916cd-c966-4ab2-b355-73409acccecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:43.386848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1074/2162] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:43.391733 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.471755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408d4ae-88da-4d75-b989-448d84248e0d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.477460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.478134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.478428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.478703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.479528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.481819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.493167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:43.511543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1089/2163] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:43 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:43.516275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee916cd-c966-4ab2-b355-73409acccecb None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.528224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee916cd-c966-4ab2-b355-73409acccecb None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.586155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.586821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.587206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.587483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.588243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.590380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.611197 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1075/2164] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:43 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:43.615617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.627203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57698129-2f10-465a-bbbb-e034f0e88b66 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.632681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.633338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.633616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.633886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.634635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.636297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.650283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1090/2165] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:43.654524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.664845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961011f-d5ae-4b4a-8888-c835cb2d1570 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.669752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf7514-3f1d-44b0-a3ae-b7c5c52fa316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.670413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7514-3f1d-44b0-a3ae-b7c5c52fa316 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.670699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7514-3f1d-44b0-a3ae-b7c5c52fa316 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.670976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7514-3f1d-44b0-a3ae-b7c5c52fa316 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.671715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7514-3f1d-44b0-a3ae-b7c5c52fa316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.673381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7514-3f1d-44b0-a3ae-b7c5c52fa316 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.678745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:43.684655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1076/2166] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:43.692183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.703802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a52581-aa3d-497e-aa6e-5e26c371fc35 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.709578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.710504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.710756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.711033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] PATH_INFO: `/v3/projects/9052274053554fa4b023539ffcb43aa3/users/128bbe40b7b0437181dc25610543f519/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.728653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9052274053554fa4b023539ffcb43aa3, user_id=128bbe40b7b0437181dc25610543f519, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.731382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:43.746181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1091/2167] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:43 2022] PUT /identity/v3/projects/9052274053554fa4b023539ffcb43aa3/users/128bbe40b7b0437181dc25610543f519/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:43.756302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf651c-c1bf-4f8a-8fc9-63f38bc04a48 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.756801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf651c-c1bf-4f8a-8fc9-63f38bc04a48 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.756920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf651c-c1bf-4f8a-8fc9-63f38bc04a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.786714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7adf651c-c1bf-4f8a-8fc9-63f38bc04a48 None None] MFA Rules not processed for user `128bbe40b7b0437181dc25610543f519`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:43.788016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7adf651c-c1bf-4f8a-8fc9-63f38bc04a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:43.859860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.864037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:43.909682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1077/2168] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:43.916569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf651c-c1bf-4f8a-8fc9-63f38bc04a48 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:43.935678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44146df2-4138-4b46-b5da-4d639bf38038 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:43.941623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:43.943247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:43.943700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:43.944265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] PATH_INFO: `/v3/users/19ee0bee9bfc4803aeb4216a598b03dc` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:43.945535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ee0bee9bfc4803aeb4216a598b03dc)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:43.947871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.015727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf651c-c1bf-4f8a-8fc9-63f38bc04a48 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.021357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.022267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.022804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.022944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.023702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.025669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.036831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ee0bee9bfc4803aeb4216a598b03dc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.047861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ee0bee9bfc4803aeb4216a598b03dc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.058017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ee0bee9bfc4803aeb4216a598b03dc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.059111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ee0bee9bfc4803aeb4216a598b03dc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.059920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ee0bee9bfc4803aeb4216a598b03dc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.065816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1078/2169] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:43 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:44.069490 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.073900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.088985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1092/2170] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:43 2022] DELETE /identity/v3/users/19ee0bee9bfc4803aeb4216a598b03dc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:44.090306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.092324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.140059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:44.150345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1beac8-70b8-4a2d-9922-1ed2435c9fd2 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.170004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12497ea6-b451-4630-a43d-178cfb4afc0f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.174920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b51263-5e71-446a-b8ae-bb841f457327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.175603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b51263-5e71-446a-b8ae-bb841f457327 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.175888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b51263-5e71-446a-b8ae-bb841f457327 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.176195 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b51263-5e71-446a-b8ae-bb841f457327 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.176894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b51263-5e71-446a-b8ae-bb841f457327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.179002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b51263-5e71-446a-b8ae-bb841f457327 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.181312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:44.206014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1093/2171] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:44 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:44.208398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0c7a-72a7-4d8c-baf6-7bdea85f1903 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.209122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.209697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0c7a-72a7-4d8c-baf6-7bdea85f1903 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.209830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.210054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.210152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a0c7a-72a7-4d8c-baf6-7bdea85f1903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.210334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.212452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.237585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3c3a0c7a-72a7-4d8c-baf6-7bdea85f1903 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:44.238811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3c3a0c7a-72a7-4d8c-baf6-7bdea85f1903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.247935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1094/2172] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:44.247935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a0c7a-72a7-4d8c-baf6-7bdea85f1903 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.261614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a0c7a-72a7-4d8c-baf6-7bdea85f1903 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.266646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.269992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.269992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.269992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.269992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.270312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.280871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1095/2173] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:44.281917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.283574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.302621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.304017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.313704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1079/2174] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:44.314475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.324246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619e9b2-24d8-40a0-a1fc-2bb3d585eb6e service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.328935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.329817 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.330102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.330376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] PATH_INFO: `/v3/projects/f9b502fe709547e1921f2352b703cd26/users/9cbd8947630c4d5587631f4bffc7d251/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.343390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd43a9-63eb-4fc4-aee9-8da644a5c42e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.347430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.348224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.348585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.348742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b502fe709547e1921f2352b703cd26, user_id=9cbd8947630c4d5587631f4bffc7d251, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.348826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.351676 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.359926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.361426 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.366746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1080/2175] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:44 2022] PUT /identity/v3/projects/f9b502fe709547e1921f2352b703cd26/users/9cbd8947630c4d5587631f4bffc7d251/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:44.372412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.374008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1096/2176] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:44.452475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dea4f3-6137-4c0b-997f-453bb96d2733 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.458100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8242911-33c6-4f9c-8d83-6e90ff2f7743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.458861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8242911-33c6-4f9c-8d83-6e90ff2f7743 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.460433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8242911-33c6-4f9c-8d83-6e90ff2f7743 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.460433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8242911-33c6-4f9c-8d83-6e90ff2f7743 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.460433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8242911-33c6-4f9c-8d83-6e90ff2f7743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.465958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8242911-33c6-4f9c-8d83-6e90ff2f7743 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.481253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1081/2177] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:44.482543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.494895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728008-d584-429e-92b3-142fbc41606d service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.501995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.504563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.505177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.506247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] PATH_INFO: `/v3/projects/f9b502fe709547e1921f2352b703cd26/users/9cbd8947630c4d5587631f4bffc7d251/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.508812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b502fe709547e1921f2352b703cd26, user_id=9cbd8947630c4d5587631f4bffc7d251, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.512219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.525214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1097/2178] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:44 2022] PUT /identity/v3/projects/f9b502fe709547e1921f2352b703cd26/users/9cbd8947630c4d5587631f4bffc7d251/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:44.535011 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b582db-1917-4911-9125-27189165d764 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.535303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b582db-1917-4911-9125-27189165d764 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.535582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b582db-1917-4911-9125-27189165d764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.565598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b9b582db-1917-4911-9125-27189165d764 None None] MFA Rules not processed for user `9cbd8947630c4d5587631f4bffc7d251`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:44.567685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b9b582db-1917-4911-9125-27189165d764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.628834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:44.664023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1082/2179] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:44.676535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.686314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b582db-1917-4911-9125-27189165d764 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.688183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b9b582db-1917-4911-9125-27189165d764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.743534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eac5c-0086-4fb1-9aef-17bc946941c4 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.747405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.748102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.748382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.748657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.750883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:44.772036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b582db-1917-4911-9125-27189165d764 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.778098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.778994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.779258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.779536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] PATH_INFO: `/v3/projects/10b990353257486e9c2539f748bf3431` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.781294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10b990353257486e9c2539f748bf3431)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.783223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.810305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:44.822142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.823527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.832067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10b990353257486e9c2539f748bf3431', 'request_id': 'req-a81417af-b857-404f-b8e4-6746199a2200'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.834092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1098/2180] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:44.840541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10b990353257486e9c2539f748bf3431', 'request_id': 'req-a81417af-b857-404f-b8e4-6746199a2200'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.846998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10b990353257486e9c2539f748bf3431', 'request_id': 'req-a81417af-b857-404f-b8e4-6746199a2200'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.857925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] The token cache is being invalidate because project 10b990353257486e9c2539f748bf3431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:44.858215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.858876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.860489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1083/2181] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:44 2022] DELETE /identity/v3/projects/10b990353257486e9c2539f748bf3431 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:44.864958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:44.875861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db0902-9379-437c-b394-cfec6178c7bf service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:44.934078 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:44.934965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:44.935262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:44.935530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] PATH_INFO: `/v3/users/19035d063cf14b46a975b4645a2a2828` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:44.936872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19035d063cf14b46a975b4645a2a2828)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:44.938802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:44.988549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19035d063cf14b46a975b4645a2a2828'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:44.999111 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19035d063cf14b46a975b4645a2a2828'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.006707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19035d063cf14b46a975b4645a2a2828'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.007366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19035d063cf14b46a975b4645a2a2828'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.008092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4d9fd9d1-c1ac-4b74-b083-79e12dd876e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19035d063cf14b46a975b4645a2a2828'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.027020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1099/2182] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:44 2022] DELETE /identity/v3/users/19035d063cf14b46a975b4645a2a2828 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:45.441072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:45.442769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:45.459487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81417af-b857-404f-b8e4-6746199a2200 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:45.558198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:45.559254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:45.559591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:45.559931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] PATH_INFO: `/v3/projects/4e19801764754ac8bcc23c5a8313ccec` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:45.561221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e19801764754ac8bcc23c5a8313ccec)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:45.563200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:45.574584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:45.618270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e19801764754ac8bcc23c5a8313ccec', 'request_id': 'req-059db129-5ef1-4254-baaa-153d63045539'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.626006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e19801764754ac8bcc23c5a8313ccec', 'request_id': 'req-059db129-5ef1-4254-baaa-153d63045539'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.632606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e19801764754ac8bcc23c5a8313ccec', 'request_id': 'req-059db129-5ef1-4254-baaa-153d63045539'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.644326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] The token cache is being invalidate because project 4e19801764754ac8bcc23c5a8313ccec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:45.644612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.645333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-059db129-5ef1-4254-baaa-153d63045539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:45.647206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1084/2183] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:45 2022] DELETE /identity/v3/projects/4e19801764754ac8bcc23c5a8313ccec => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:45.651967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174ae1c-940b-4909-aeb4-2ec5485873e6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:45.652326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174ae1c-940b-4909-aeb4-2ec5485873e6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:45.652609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174ae1c-940b-4909-aeb4-2ec5485873e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:45.681168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-b174ae1c-940b-4909-aeb4-2ec5485873e6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:45.682399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b174ae1c-940b-4909-aeb4-2ec5485873e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:45.795099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80484e05-a56b-49b8-a771-7fbe84333461 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:45.795641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80484e05-a56b-49b8-a771-7fbe84333461 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:45.795771 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80484e05-a56b-49b8-a771-7fbe84333461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:45.800193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1100/2184] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:45.817117 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-80484e05-a56b-49b8-a771-7fbe84333461 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:45.818398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-80484e05-a56b-49b8-a771-7fbe84333461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:45.836052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174ae1c-940b-4909-aeb4-2ec5485873e6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:45.836269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1085/2185] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:45.841955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174ae1c-940b-4909-aeb4-2ec5485873e6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:45.846478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:45.847131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:45.847414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:45.847689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:45.848472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:45.848780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:45.850472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:45.858040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:45.867672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1101/2186] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:45.869093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80484e05-a56b-49b8-a771-7fbe84333461 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:45.879769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80484e05-a56b-49b8-a771-7fbe84333461 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:45.885043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:45.885708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:45.885987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:45.886262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:45.886996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:45.889154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:45.916621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1086/2187] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:45 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:45.920579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:45.934817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d87eb85-29a8-46c3-b5df-ecc3054d45d0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:46.007794 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:46.008597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:46.008879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:46.009154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:46.009867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:46.011885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:46.035776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1102/2188] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:45 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:46.039797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:46.050737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77073188-9818-42e6-9f8a-ae298341e324 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:46.055328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:46.056046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:46.056326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:46.056596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:46.057347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:46.059004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:46.069990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1087/2189] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:46.074039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:46.084145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b63107-db5d-4ab3-be0b-2e1847e192b4 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:46.089267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1269f60-db4f-4c93-a810-6627862ec8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:46.089912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1269f60-db4f-4c93-a810-6627862ec8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:46.090230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1269f60-db4f-4c93-a810-6627862ec8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:46.090539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1269f60-db4f-4c93-a810-6627862ec8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:46.091361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1269f60-db4f-4c93-a810-6627862ec8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:46.093008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1269f60-db4f-4c93-a810-6627862ec8a6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:46.103343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1103/2190] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:46.107438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:46.118781 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ecd9a-3d27-4810-b33f-2e5ee92aa440 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:46.124910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:46.125746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:46.126028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:46.126305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] PATH_INFO: `/v3/projects/27fd3f12712e4cf28dc4e13b0bc514ae/users/be9cc42ff7854673a7ee9f36dd4fe8f4/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:46.141596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27fd3f12712e4cf28dc4e13b0bc514ae, user_id=be9cc42ff7854673a7ee9f36dd4fe8f4, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:46.144302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:46.158427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1088/2191] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:46 2022] PUT /identity/v3/projects/27fd3f12712e4cf28dc4e13b0bc514ae/users/be9cc42ff7854673a7ee9f36dd4fe8f4/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:46.170060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:46.170465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:46.170805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:46.207565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] MFA Rules not processed for user `be9cc42ff7854673a7ee9f36dd4fe8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:46.208815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:46.290707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1104/2192] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:46.301758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:46.303581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:46.343583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:46.345036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:46.361853 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e378da48-5a7b-4cf6-bac7-b4be8713d5b3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:46.366709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:46.367368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:46.367702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:46.367938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:46.376851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:46.378748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:46.387899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1089/2193] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:46.402300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a8b0d-c4d6-4150-a231-de255a6ccbe1 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:46.405992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:46.406637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:46.406918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:46.407194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:46.409058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:46.432117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:46.479118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:46.480913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:46.491100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1105/2194] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:47.276144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:47.276144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8740b2-919c-4114-9253-9bd158c27a89 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:47.278393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ac3d2-cffd-476e-94a5-06f2d011ca09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:47.279158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ac3d2-cffd-476e-94a5-06f2d011ca09 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:47.279472 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ac3d2-cffd-476e-94a5-06f2d011ca09 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:47.279873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ac3d2-cffd-476e-94a5-06f2d011ca09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:47.281969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-425ac3d2-cffd-476e-94a5-06f2d011ca09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:47.321829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:47.353759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ac3d2-cffd-476e-94a5-06f2d011ca09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:47.355390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ac3d2-cffd-476e-94a5-06f2d011ca09 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:47.365572 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1090/2195] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:48.374255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:48.376482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:48.446076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e645a-b1af-42e3-850b-fb7275c5fe5d service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:48.450507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:48.451377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:48.451679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:48.451947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] PATH_INFO: `/v3/users/c83c0955b7794d7393d09d5b04185897` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:48.453179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83c0955b7794d7393d09d5b04185897)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:48.455102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:48.504705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83c0955b7794d7393d09d5b04185897'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:48.513605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83c0955b7794d7393d09d5b04185897'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:48.521059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83c0955b7794d7393d09d5b04185897'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:48.521709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83c0955b7794d7393d09d5b04185897'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:48.522312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83c0955b7794d7393d09d5b04185897'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:48.540313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1106/2196] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:48 2022] DELETE /identity/v3/users/c83c0955b7794d7393d09d5b04185897 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:48.545132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:48.545646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:48.545740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:48.564471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:48.565727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:48.625903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1091/2197] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:48.635815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:48.637277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:48.691603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:48.710023 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbec3b1-17db-48d4-b87e-acc6234fb1fc admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:48.715336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:48.716182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:48.716553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:48.716893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:48.733369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:48.736313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:48.748864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1107/2198] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:49.053465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:49.055085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:49.069348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64da9b-4bb0-43ca-be61-2f00737eee0e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:49.074110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:49.074984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:49.075262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:49.075533 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] PATH_INFO: `/v3/projects/272515157873449cbe8ed5b774421c96` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:49.076762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=272515157873449cbe8ed5b774421c96)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:49.078673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:49.128311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '272515157873449cbe8ed5b774421c96', 'request_id': 'req-67e68d6b-62bd-45a2-b9f9-b62c08935956'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.137324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '272515157873449cbe8ed5b774421c96', 'request_id': 'req-67e68d6b-62bd-45a2-b9f9-b62c08935956'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.143616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '272515157873449cbe8ed5b774421c96', 'request_id': 'req-67e68d6b-62bd-45a2-b9f9-b62c08935956'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.154408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] The token cache is being invalidate because project 272515157873449cbe8ed5b774421c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:49.154696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.155335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.156938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1092/2199] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:49 2022] DELETE /identity/v3/projects/272515157873449cbe8ed5b774421c96 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:49.161076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:49.172281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c90d8-fd18-43df-a941-084752d5d6d1 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:49.234787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:49.237106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:49.237106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:49.237106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] PATH_INFO: `/v3/users/2201396db9c14467b0fa24dd72b02c75` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:49.237106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2201396db9c14467b0fa24dd72b02c75)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:49.238694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:49.306540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2201396db9c14467b0fa24dd72b02c75'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.317260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2201396db9c14467b0fa24dd72b02c75'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.326394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2201396db9c14467b0fa24dd72b02c75'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.327456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2201396db9c14467b0fa24dd72b02c75'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.328277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2201396db9c14467b0fa24dd72b02c75'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.377883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1108/2200] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:49 2022] DELETE /identity/v3/users/2201396db9c14467b0fa24dd72b02c75 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:49.734210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:49.735800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:49.764866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e68d6b-62bd-45a2-b9f9-b62c08935956 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:49.826852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:49.845563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:49.846612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:49.847185 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:49.847810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] PATH_INFO: `/v3/projects/8fc7a9e82ea94ce7b0cefcf840f723b0` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:49.849408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc7a9e82ea94ce7b0cefcf840f723b0)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:49.851652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:49.902991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc7a9e82ea94ce7b0cefcf840f723b0', 'request_id': 'req-93027ced-f247-486f-89f0-166575fb62b2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.911506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc7a9e82ea94ce7b0cefcf840f723b0', 'request_id': 'req-93027ced-f247-486f-89f0-166575fb62b2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.918178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc7a9e82ea94ce7b0cefcf840f723b0', 'request_id': 'req-93027ced-f247-486f-89f0-166575fb62b2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.929302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] The token cache is being invalidate because project 8fc7a9e82ea94ce7b0cefcf840f723b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:49.929591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.930310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:49.932054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1093/2201] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:49 2022] DELETE /identity/v3/projects/8fc7a9e82ea94ce7b0cefcf840f723b0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:49.936872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:49.939392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:49.956542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5764f0-f337-4e9d-b41a-fa38f9f9d531 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:50.018243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:50.019128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:50.019409 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:50.019682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] PATH_INFO: `/v3/users/4765b09fb948458ca4200ab1815d8e44` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:50.021051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4765b09fb948458ca4200ab1815d8e44)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:50.022827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:50.026522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:50.078374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4765b09fb948458ca4200ab1815d8e44'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.086563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4765b09fb948458ca4200ab1815d8e44'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.093656 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4765b09fb948458ca4200ab1815d8e44'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.094324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4765b09fb948458ca4200ab1815d8e44'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.094929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7e0472a3-082c-400d-af65-520c82fb566b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4765b09fb948458ca4200ab1815d8e44'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.116611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1109/2202] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:49 2022] DELETE /identity/v3/users/4765b09fb948458ca4200ab1815d8e44 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:50.405321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:50.407019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:50.420960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027ced-f247-486f-89f0-166575fb62b2 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:50.483386 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:50.484304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:50.484589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:50.484865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] PATH_INFO: `/v3/projects/33cb711c87da4009969876ca5455041b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:50.486120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cb711c87da4009969876ca5455041b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:50.488199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:50.539104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cb711c87da4009969876ca5455041b', 'request_id': 'req-f1fa626a-cd92-422c-a13d-18c351832d6a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.549459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cb711c87da4009969876ca5455041b', 'request_id': 'req-f1fa626a-cd92-422c-a13d-18c351832d6a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.556462 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cb711c87da4009969876ca5455041b', 'request_id': 'req-f1fa626a-cd92-422c-a13d-18c351832d6a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.569069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] The token cache is being invalidate because project 33cb711c87da4009969876ca5455041b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:50.569358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.570036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1fa626a-cd92-422c-a13d-18c351832d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:50.571606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1094/2203] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:50 2022] DELETE /identity/v3/projects/33cb711c87da4009969876ca5455041b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:50.576778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcadf1b-7643-40dc-b7c5-fa246f12f8d8 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:50.577245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcadf1b-7643-40dc-b7c5-fa246f12f8d8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:50.577375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcadf1b-7643-40dc-b7c5-fa246f12f8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:50.599343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7bcadf1b-7643-40dc-b7c5-fa246f12f8d8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:50.600648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7bcadf1b-7643-40dc-b7c5-fa246f12f8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:50.709208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1110/2204] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:50.718110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c951d32-5c49-496f-bcf6-a4802f5fea64 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:50.718534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c951d32-5c49-496f-bcf6-a4802f5fea64 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:50.718900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c951d32-5c49-496f-bcf6-a4802f5fea64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:50.744528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4c951d32-5c49-496f-bcf6-a4802f5fea64 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:50.746026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4c951d32-5c49-496f-bcf6-a4802f5fea64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:50.755069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1095/2205] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:50.759958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcadf1b-7643-40dc-b7c5-fa246f12f8d8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:50.770490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcadf1b-7643-40dc-b7c5-fa246f12f8d8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:50.777178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:50.777984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:50.778363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:50.778731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:50.779555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:50.779976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:50.783485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:50.806233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1111/2206] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:50.810336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c951d32-5c49-496f-bcf6-a4802f5fea64 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:50.825015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c951d32-5c49-496f-bcf6-a4802f5fea64 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:50.830426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:50.831348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:50.831778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:50.832226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:50.833225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:50.836070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:50.840945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:50.875709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1096/2207] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:50 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:50.876922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:50.887638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae344d-832b-4f2d-808d-db9fe8a0fb96 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:50.956107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:50.956780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:50.957060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:50.957332 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:50.958041 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:50.959953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:50.976506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:50.987500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1112/2208] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:50 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:50.992294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:51.005171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a6d9d-dca2-41ad-ad69-5827cd9c6290 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:51.009926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:51.010602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:51.010883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:51.011158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:51.011970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:51.013602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:51.024722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1097/2209] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:51.028813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:51.039201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12a1a0-ae3f-4a30-9381-3df4d5aa2145 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:51.048597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288b663-cf93-4469-ab69-4ae452410127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:51.048597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b663-cf93-4469-ab69-4ae452410127 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:51.048597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b663-cf93-4469-ab69-4ae452410127 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:51.048597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288b663-cf93-4469-ab69-4ae452410127 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:51.048937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288b663-cf93-4469-ab69-4ae452410127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:51.048937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288b663-cf93-4469-ab69-4ae452410127 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:51.060579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1113/2210] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:51.064906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:51.075743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23d01e-ac10-47a2-a8f9-d9b018bf84ae admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:51.081002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:51.081895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:51.082173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:51.082448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] PATH_INFO: `/v3/projects/61bf98ebe94b421a9cd95e60981cb636/users/76f38dea87644ce292d649ed8e3d7df2/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:51.101218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61bf98ebe94b421a9cd95e60981cb636, user_id=76f38dea87644ce292d649ed8e3d7df2, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:51.104278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:51.125904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1098/2211] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:51 2022] PUT /identity/v3/projects/61bf98ebe94b421a9cd95e60981cb636/users/76f38dea87644ce292d649ed8e3d7df2/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:51.140828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:51.141203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:51.141579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:51.173421 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] MFA Rules not processed for user `76f38dea87644ce292d649ed8e3d7df2`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:51.175178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:51.256148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1114/2212] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:51.271001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:51.275005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:51.358973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210e527-7940-4071-a202-0016daa65eb3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:51.365134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4095d1-e0dc-4688-8672-44ed74b9d9f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:51.366485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4095d1-e0dc-4688-8672-44ed74b9d9f2 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:51.366832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4095d1-e0dc-4688-8672-44ed74b9d9f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:51.367159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4095d1-e0dc-4688-8672-44ed74b9d9f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:51.377392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4095d1-e0dc-4688-8672-44ed74b9d9f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:51.379001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4095d1-e0dc-4688-8672-44ed74b9d9f2 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:51.389774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1099/2213] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:51.558520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:51.561223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:51.692143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57063a6-6b1f-4eb2-8303-6a9b31c91222 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:51.697181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:51.698097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:51.698372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:51.698638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] PATH_INFO: `/v3/users/7378d0a9a8b84a5783b0f8267de64e63` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:51.700070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7378d0a9a8b84a5783b0f8267de64e63)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:51.702354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:51.809109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7378d0a9a8b84a5783b0f8267de64e63'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:51.842760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7378d0a9a8b84a5783b0f8267de64e63'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:51.857237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7378d0a9a8b84a5783b0f8267de64e63'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:51.868862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7378d0a9a8b84a5783b0f8267de64e63'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:51.870251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7378d0a9a8b84a5783b0f8267de64e63'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:51.910875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:51.927832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1115/2214] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:51 2022] DELETE /identity/v3/users/7378d0a9a8b84a5783b0f8267de64e63 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:51.932699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:51.933182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:51.933318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:51.964851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:51.966596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:52.064403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1100/2215] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:52.083202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:52.085286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:52.179393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd69a9e-2ea1-4d3c-9518-37e238b325a1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:52.184274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:52.184985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:52.185269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:52.185541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:52.196925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:52.198396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:52.218926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1116/2216] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:52.641342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:52.642882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:52.663948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3c2d0-b0f1-42cb-a25c-ee6b17aef77f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:52.669190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:52.670452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:52.670730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:52.672272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] PATH_INFO: `/v3/projects/c5e79a83ac1f481f8544b6b1031fe006` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:52.674206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e79a83ac1f481f8544b6b1031fe006)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:52.676498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:52.778878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e79a83ac1f481f8544b6b1031fe006', 'request_id': 'req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:52.781028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:52.793110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e79a83ac1f481f8544b6b1031fe006', 'request_id': 'req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:52.821699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e79a83ac1f481f8544b6b1031fe006', 'request_id': 'req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:52.862962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] The token cache is being invalidate because project c5e79a83ac1f481f8544b6b1031fe006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:52.863259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:52.865947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:52.869149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1101/2217] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:52 2022] DELETE /identity/v3/projects/c5e79a83ac1f481f8544b6b1031fe006 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:52.873115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:52.887714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77168197-cdd9-401b-af99-fae8afa5a1f8 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:52.997674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:52.999099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:52.999393 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:52.999650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] PATH_INFO: `/v3/users/128bbe40b7b0437181dc25610543f519` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:53.001155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=128bbe40b7b0437181dc25610543f519)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:53.003340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:53.102988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128bbe40b7b0437181dc25610543f519'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.123647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128bbe40b7b0437181dc25610543f519'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.136388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128bbe40b7b0437181dc25610543f519'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.137321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128bbe40b7b0437181dc25610543f519'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.137997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128bbe40b7b0437181dc25610543f519'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.182300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1117/2218] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:52 2022] DELETE /identity/v3/users/128bbe40b7b0437181dc25610543f519 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:53.620691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:53.622305 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:53.641985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c483e4d-0653-48eb-bc67-c9b56bfb5f04 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:53.742365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:53.743355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:53.743710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:53.744086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] PATH_INFO: `/v3/projects/9052274053554fa4b023539ffcb43aa3` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:53.745501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9052274053554fa4b023539ffcb43aa3)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:53.747709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:53.816881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9052274053554fa4b023539ffcb43aa3', 'request_id': 'req-0fa83987-a2ea-45c3-9e6b-585bc08df88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.827695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9052274053554fa4b023539ffcb43aa3', 'request_id': 'req-0fa83987-a2ea-45c3-9e6b-585bc08df88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.836495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9052274053554fa4b023539ffcb43aa3', 'request_id': 'req-0fa83987-a2ea-45c3-9e6b-585bc08df88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.855068 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] The token cache is being invalidate because project 9052274053554fa4b023539ffcb43aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:53.855368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.856061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:53.857899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1102/2219] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:53 2022] DELETE /identity/v3/projects/9052274053554fa4b023539ffcb43aa3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:53.862138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:53.867102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:53.883423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e39d5-6867-4953-85b2-a032a04b1805 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:53.907696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:53.963924 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:53.964894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:53.965282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:53.965515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] PATH_INFO: `/v3/users/9cbd8947630c4d5587631f4bffc7d251` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:53.966684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cbd8947630c4d5587631f4bffc7d251)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:53.968661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:54.022993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cbd8947630c4d5587631f4bffc7d251'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.030762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cbd8947630c4d5587631f4bffc7d251'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.037598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cbd8947630c4d5587631f4bffc7d251'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.038310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cbd8947630c4d5587631f4bffc7d251'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.038915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cbd8947630c4d5587631f4bffc7d251'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.060477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1118/2220] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:53 2022] DELETE /identity/v3/users/9cbd8947630c4d5587631f4bffc7d251 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:54.257430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:54.258934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:54.286692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83987-a2ea-45c3-9e6b-585bc08df88a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:54.377643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:54.378761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.379191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.379635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] PATH_INFO: `/v3/users/76f38dea87644ce292d649ed8e3d7df2` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.381611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] RBAC: Authorizing `identity:delete_user(user_id=76f38dea87644ce292d649ed8e3d7df2)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:54.383967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:54.395880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:54.452513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76f38dea87644ce292d649ed8e3d7df2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.453348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76f38dea87644ce292d649ed8e3d7df2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.462953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:54.464694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:54.470952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76f38dea87644ce292d649ed8e3d7df2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.480124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d909-a19e-46c2-8022-58c3b737f9fa admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:54.480995 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76f38dea87644ce292d649ed8e3d7df2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.481935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-dc1510cf-dda5-463c-9c14-c255b58b769f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76f38dea87644ce292d649ed8e3d7df2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.489171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:54.490234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.490623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.491019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] PATH_INFO: `/v3/projects/f9b502fe709547e1921f2352b703cd26` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.492482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b502fe709547e1921f2352b703cd26)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:54.499889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:54.516089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1103/2221] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:54 2022] DELETE /identity/v3/users/76f38dea87644ce292d649ed8e3d7df2 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:54.517614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.517908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.518184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.541332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:54.542733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:54.586098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b502fe709547e1921f2352b703cd26', 'request_id': 'req-46067443-9260-4c9c-83fc-5681aa37d2a1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.592934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b502fe709547e1921f2352b703cd26', 'request_id': 'req-46067443-9260-4c9c-83fc-5681aa37d2a1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.604272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b502fe709547e1921f2352b703cd26', 'request_id': 'req-46067443-9260-4c9c-83fc-5681aa37d2a1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.617456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] The token cache is being invalidate because project f9b502fe709547e1921f2352b703cd26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:54.617744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.618376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-46067443-9260-4c9c-83fc-5681aa37d2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:54.620275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1119/2222] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:54 2022] DELETE /identity/v3/projects/f9b502fe709547e1921f2352b703cd26 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:54.624296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1104/2223] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:54.626097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c61f5-f31d-4c40-8972-85800327448c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.626847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c61f5-f31d-4c40-8972-85800327448c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.628386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77c61f5-f31d-4c40-8972-85800327448c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.635296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:54.637062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:54.651510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f77c61f5-f31d-4c40-8972-85800327448c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:54.653042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f77c61f5-f31d-4c40-8972-85800327448c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:54.672272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:54.709353 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60b6b6-3fd9-4658-9e3a-c0e3478be806 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:54.732477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1120/2224] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:54.743517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1ac2-be0e-4666-8fbc-2dd044550f3d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.743787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1ac2-be0e-4666-8fbc-2dd044550f3d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.744089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1ac2-be0e-4666-8fbc-2dd044550f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.770205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-780b1ac2-be0e-4666-8fbc-2dd044550f3d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:54.773992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-780b1ac2-be0e-4666-8fbc-2dd044550f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:54.785346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1121/2225] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:54.789709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1ac2-be0e-4666-8fbc-2dd044550f3d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:54.795223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:54.803716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1ac2-be0e-4666-8fbc-2dd044550f3d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:54.811626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:54.812531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.812941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.813311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.814153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:54.814564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:54.817206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:54.818017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:54.818704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.819236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.819522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.823907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:54.833330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1122/2226] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:54.837571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:54.847951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:54.849232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f21400-8d9a-4b37-a77c-b80a806b5946 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:54.850668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:54.854484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:54.855297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:54.855646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:54.856006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:54.856985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:54.860449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:54.868020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1105/2227] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:54.891229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1123/2228] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:54 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:54.894449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:54.904549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1ef59-5089-4482-8b3a-212099597837 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:55.008585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:55.014694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:55.015517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:55.015960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:55.016773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:55.020256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:55.066431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1106/2229] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:54 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:55.070374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:55.083565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd729f06-1d77-4e2a-933b-a23a58486de7 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:55.089264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:55.090105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:55.090392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:55.090666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:55.091400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:55.093067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:55.110665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1124/2230] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:55.114760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:55.132481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f60980-ae8a-4d79-88a5-ed7871388323 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:55.138112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c7e4f-391e-468d-94ee-6064b3731488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:55.138884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7e4f-391e-468d-94ee-6064b3731488 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:55.139135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7e4f-391e-468d-94ee-6064b3731488 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:55.139422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c7e4f-391e-468d-94ee-6064b3731488 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:55.140393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c7e4f-391e-468d-94ee-6064b3731488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:55.142396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c7e4f-391e-468d-94ee-6064b3731488 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:55.163309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1107/2231] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:55.163622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:55.176721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf2a1e-36a4-4648-816b-1f0490807ccb admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:55.182304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:55.183264 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:55.183521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:55.183796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] PATH_INFO: `/v3/projects/5ac1871137494335b9cee22170d3315b/users/f2b4682563a24786804dee37d5ee31eb/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:55.203708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac1871137494335b9cee22170d3315b, user_id=f2b4682563a24786804dee37d5ee31eb, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:55.207413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:55.254722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1125/2232] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:55 2022] PUT /identity/v3/projects/5ac1871137494335b9cee22170d3315b/users/f2b4682563a24786804dee37d5ee31eb/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:55.257258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:55.258177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:55.258427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:55.322851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] MFA Rules not processed for user `f2b4682563a24786804dee37d5ee31eb`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:55.324437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:55.410846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1108/2233] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:55.422205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:55.423913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:55.490893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbceecb3-b547-4fca-82e1-c83a5aaa7cc6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:55.493703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660a7e-53d7-4a36-98d2-b6aadd267f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:55.494359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660a7e-53d7-4a36-98d2-b6aadd267f64 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:55.494790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660a7e-53d7-4a36-98d2-b6aadd267f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:55.495063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660a7e-53d7-4a36-98d2-b6aadd267f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:55.504632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660a7e-53d7-4a36-98d2-b6aadd267f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:55.506116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660a7e-53d7-4a36-98d2-b6aadd267f64 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:55.516431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1126/2234] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:55.764447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:55.766823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:55.895961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:55.903430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95934e2-9e0d-455c-bc05-0af96973ed45 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:55.909036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:55.910379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:55.910379 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:55.910609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] PATH_INFO: `/v3/projects/61bf98ebe94b421a9cd95e60981cb636` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:55.911780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61bf98ebe94b421a9cd95e60981cb636)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:55.913737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:55.973943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61bf98ebe94b421a9cd95e60981cb636', 'request_id': 'req-d34c0594-4e09-4eba-817e-f62890d7ba89'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:55.983639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61bf98ebe94b421a9cd95e60981cb636', 'request_id': 'req-d34c0594-4e09-4eba-817e-f62890d7ba89'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:55.992337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61bf98ebe94b421a9cd95e60981cb636', 'request_id': 'req-d34c0594-4e09-4eba-817e-f62890d7ba89'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.008276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] The token cache is being invalidate because project 61bf98ebe94b421a9cd95e60981cb636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:56.008669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.009352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.011785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1109/2235] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:55 2022] DELETE /identity/v3/projects/61bf98ebe94b421a9cd95e60981cb636 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:56.016844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035b3bf-5588-4db8-8569-da403682a21e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.017311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035b3bf-5588-4db8-8569-da403682a21e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.017387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035b3bf-5588-4db8-8569-da403682a21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.035800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.037690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:56.068493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6035b3bf-5588-4db8-8569-da403682a21e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:56.070214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6035b3bf-5588-4db8-8569-da403682a21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:56.162133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c0594-4e09-4eba-817e-f62890d7ba89 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.167073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.168256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.168656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.169003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] PATH_INFO: `/v3/users/ead6569ad30b496092f49b3fe0c8a706` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.170331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead6569ad30b496092f49b3fe0c8a706)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.172611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.181341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:56.223124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1127/2236] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:56.255863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead6569ad30b496092f49b3fe0c8a706'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.256623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead6569ad30b496092f49b3fe0c8a706'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.280680 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead6569ad30b496092f49b3fe0c8a706'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.294878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead6569ad30b496092f49b3fe0c8a706'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.295950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead6569ad30b496092f49b3fe0c8a706'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.300541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cb8ee-7d46-43c4-ac1d-18f285e33295 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.300830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cb8ee-7d46-43c4-ac1d-18f285e33295 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.301124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cb8ee-7d46-43c4-ac1d-18f285e33295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.316841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1110/2237] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:56 2022] DELETE /identity/v3/users/ead6569ad30b496092f49b3fe0c8a706 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:56.340758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a98cb8ee-7d46-43c4-ac1d-18f285e33295 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:56.342005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a98cb8ee-7d46-43c4-ac1d-18f285e33295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:56.410139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1128/2238] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:56.414530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.422492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:56.426942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954fea1-83b8-47eb-9d28-911f3da9abd3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.435232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.436166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.436679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.436987 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.437920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:56.438739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.440796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.460097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1111/2239] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:51:56.462798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cb8ee-7d46-43c4-ac1d-18f285e33295 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.473350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cb8ee-7d46-43c4-ac1d-18f285e33295 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.478300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.478951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.479245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.479515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.480337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.482956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.515660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.517582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:56.527736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1129/2240] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/projects => generated 359 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:56.537352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07ac090-41f9-4d5f-8e00-e6625a77ae4b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.537528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.617807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.620366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.620793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.621148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] PATH_INFO: `/v3/projects/4b65df0cd27d479e9bc39ecb95ff04d7` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.622656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b65df0cd27d479e9bc39ecb95ff04d7)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.625295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.660639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e19e-4c27-4c87-bb06-56d71ea15764 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.666985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.667787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.668607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.668889 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.669695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.671684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.692157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b65df0cd27d479e9bc39ecb95ff04d7', 'request_id': 'req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.703407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b65df0cd27d479e9bc39ecb95ff04d7', 'request_id': 'req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.708193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1130/2241] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/users => generated 280 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:56.713341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.717636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b65df0cd27d479e9bc39ecb95ff04d7', 'request_id': 'req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.727675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03b0d1-58a5-4654-9153-f9f061b364ad admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.735551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] The token cache is being invalidate because project 4b65df0cd27d479e9bc39ecb95ff04d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:56.736047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.737046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-48e86836-f7e7-4808-9f1b-28db4c0d9b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:56.744584 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1112/2242] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:56 2022] DELETE /identity/v3/projects/4b65df0cd27d479e9bc39ecb95ff04d7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:56.746614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497047-c0ad-4508-a461-188b08cd43e5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.746907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497047-c0ad-4508-a461-188b08cd43e5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.747182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497047-c0ad-4508-a461-188b08cd43e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.766837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-01497047-c0ad-4508-a461-188b08cd43e5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:56.768221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-01497047-c0ad-4508-a461-188b08cd43e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:56.801546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:56.810587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.811413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.811790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.812193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.813042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.816036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.829091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1131/2243] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:56.833811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.835381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:56.847814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1113/2244] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:56.850408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09441792-a9f0-456e-a786-fc28bad8909b admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.859398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a531-db3b-44eb-9b62-d7edf85a633e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.860102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a531-db3b-44eb-9b62-d7edf85a633e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.860388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a531-db3b-44eb-9b62-d7edf85a633e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.861075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.862127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.862537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.862932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.864082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.866223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.889826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1132/2245] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:56.890006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c914a531-db3b-44eb-9b62-d7edf85a633e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:56.890849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.891590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c914a531-db3b-44eb-9b62-d7edf85a633e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:56.901250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1114/2246] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:51:56.903827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b8479-c372-4d00-b83c-f12d9bc7c144 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.905496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914a531-db3b-44eb-9b62-d7edf85a633e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.908875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.909795 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.910182 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.910338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] PATH_INFO: `/v3/projects/6a0d5c681e2449999e5c5829ff38945a/users/9d5e16507d3b490995eed013ca606554/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.917699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914a531-db3b-44eb-9b62-d7edf85a633e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:56.924161 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01fa4a-d0ee-488c-9f7f-54f1532abb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:56.924841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01fa4a-d0ee-488c-9f7f-54f1532abb2b admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.925281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01fa4a-d0ee-488c-9f7f-54f1532abb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.925556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01fa4a-d0ee-488c-9f7f-54f1532abb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.926410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01fa4a-d0ee-488c-9f7f-54f1532abb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:51:56.926596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01fa4a-d0ee-488c-9f7f-54f1532abb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.928316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01fa4a-d0ee-488c-9f7f-54f1532abb2b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.930597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0d5c681e2449999e5c5829ff38945a, user_id=9d5e16507d3b490995eed013ca606554, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:56.934360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:56.952052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1115/2247] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:51:56.952052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2ec4c-d5df-4739-9b81-f05e272edd93 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:56.952650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2ec4c-d5df-4739-9b81-f05e272edd93 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:56.952985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2ec4c-d5df-4739-9b81-f05e272edd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:56.954658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1133/2248] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:56 2022] PUT /identity/v3/projects/6a0d5c681e2449999e5c5829ff38945a/users/9d5e16507d3b490995eed013ca606554/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:56.955589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:56.960257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:56.978317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3ea2ec4c-d5df-4739-9b81-f05e272edd93 None None] MFA Rules not processed for user `f2b4682563a24786804dee37d5ee31eb`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:56.979697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3ea2ec4c-d5df-4739-9b81-f05e272edd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.027884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c9c07-24f4-4916-9f51-29f8824912a0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.034114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.034784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.035065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.035362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.036102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.038070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.047516 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1116/2249] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:57.053137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.053595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.053960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.068552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1134/2250] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:56 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:57.073247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.087813 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] MFA Rules not processed for user `9d5e16507d3b490995eed013ca606554`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:57.089705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.153675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb12c97-6e12-4da2-96e7-21ce92aafa7d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.158814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.159499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.159781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.160086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.160796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.162961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.166837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1117/2251] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:57 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:57.178969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.180426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.188662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1135/2252] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:57 2022] POST /identity/v3/users => generated 303 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:57.192579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.202984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f29318-58c8-43b0-a705-3603da53ba0f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.207838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.208549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.208826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.209106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.209960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.211600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.224021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1136/2253] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:57.226793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.243583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6e93f-7d45-4623-971a-bc8b4232a8aa admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.249397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.250543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.251659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.251659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.251659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.253531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.255109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7540ef-aae5-444d-b545-3b9da5bee7b8 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.258851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1b597-a202-4580-8492-07afabba506c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.259524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b597-a202-4580-8492-07afabba506c service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.259782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b597-a202-4580-8492-07afabba506c service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.260091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b597-a202-4580-8492-07afabba506c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.269438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1b597-a202-4580-8492-07afabba506c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.270906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1b597-a202-4580-8492-07afabba506c service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.272566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1137/2254] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:57.276602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.285341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1118/2255] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:57 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:51:57.290025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d6e9a-91fe-4da8-b22c-79c9bcece88f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.295589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.296520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.296804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.297081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] PATH_INFO: `/v3/projects/03a5f3a40398462fa1636ff28ce278e8/users/40138e6c4e7d4ccc8e25a9993b0877da/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.300630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:57.314416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a5f3a40398462fa1636ff28ce278e8, user_id=40138e6c4e7d4ccc8e25a9993b0877da, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.317131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.331658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1138/2256] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:57 2022] PUT /identity/v3/projects/03a5f3a40398462fa1636ff28ce278e8/users/40138e6c4e7d4ccc8e25a9993b0877da/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:57.342119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88984318-1be7-491e-8c19-148cb8e99a86 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.342268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88984318-1be7-491e-8c19-148cb8e99a86 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.342545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88984318-1be7-491e-8c19-148cb8e99a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.365276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.367475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.383210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-88984318-1be7-491e-8c19-148cb8e99a86 None None] MFA Rules not processed for user `40138e6c4e7d4ccc8e25a9993b0877da`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:57.384487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-88984318-1be7-491e-8c19-148cb8e99a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.433982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:57.450074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc4a5c8-01db-42f1-9eab-c3ff7259e5df admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.457124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.458039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.458523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.459114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] PATH_INFO: `/v3/users/be9cc42ff7854673a7ee9f36dd4fe8f4` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.460122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be9cc42ff7854673a7ee9f36dd4fe8f4)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.462752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.482911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1119/2257] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:57 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:57.488067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88984318-1be7-491e-8c19-148cb8e99a86 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.502190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88984318-1be7-491e-8c19-148cb8e99a86 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.508171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.509682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.510326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.510733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.511742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.514223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.537772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be9cc42ff7854673a7ee9f36dd4fe8f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:57.547705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be9cc42ff7854673a7ee9f36dd4fe8f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:57.554024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1120/2258] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:51:57 2022] POST /identity/v3/projects => generated 405 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:51:57.557825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be9cc42ff7854673a7ee9f36dd4fe8f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:57.558692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be9cc42ff7854673a7ee9f36dd4fe8f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:57.559116 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.559483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3f2eed6-2c96-4119-8fdb-2448e9626d03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be9cc42ff7854673a7ee9f36dd4fe8f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:57.563109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.628071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1139/2259] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:57 2022] DELETE /identity/v3/users/be9cc42ff7854673a7ee9f36dd4fe8f4 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:57.633504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bd80d-c76a-4c9b-9ab2-b83301d1c41b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.634585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bd80d-c76a-4c9b-9ab2-b83301d1c41b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.635014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7bd80d-c76a-4c9b-9ab2-b83301d1c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.660421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef99463-9821-4101-b04e-4b3adb46dc37 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.675711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-da7bd80d-c76a-4c9b-9ab2-b83301d1c41b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:57.677107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-da7bd80d-c76a-4c9b-9ab2-b83301d1c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.762643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.763333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.763627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.763879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.764670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.766691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.767460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1140/2260] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:57.769132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f61a4-f2b3-46e3-8d22-46c3e2e769d9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.769568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f61a4-f2b3-46e3-8d22-46c3e2e769d9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.769952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f61a4-f2b3-46e3-8d22-46c3e2e769d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.784386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:57.795910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1121/2261] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:51:57 2022] POST /identity/v3/users => generated 309 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:51:57.796878 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.801054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.809125 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9a4f61a4-f2b3-46e3-8d22-46c3e2e769d9 None None] MFA Rules not processed for user `17123d3e5fe945bbb1cb835f154965d4`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:57.810836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9a4f61a4-f2b3-46e3-8d22-46c3e2e769d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:57.863131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:57.879617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e242a-8317-4f49-b705-4a34e363eb30 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.887154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.887154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.887154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.887154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.899332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.901159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.913626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1122/2262] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:57.914755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.914971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1141/2263] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:57.928626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491d2b-d128-47d3-9d2f-47f93f4d0dd8 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.935749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.936724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.937990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.938447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:57.940055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:57.942166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:57.957421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1123/2264] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:51:57.961991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f61a4-f2b3-46e3-8d22-46c3e2e769d9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:57.974479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f61a4-f2b3-46e3-8d22-46c3e2e769d9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:57.981420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:57.982244 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:57.982524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:57.982802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] PATH_INFO: `/v3/projects/e217f3625a58433cabf6d750a17e8039/users/a979fd878af04915b18a1ed246dc32e0/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.001200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e217f3625a58433cabf6d750a17e8039, user_id=a979fd878af04915b18a1ed246dc32e0, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:58.004110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:58.025376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1142/2265] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:57 2022] PUT /identity/v3/projects/e217f3625a58433cabf6d750a17e8039/users/a979fd878af04915b18a1ed246dc32e0/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:58.029806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:58.043459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d4faed-eeb7-4c82-90bb-37f66fdf124b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:58.123263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da6abc-a90b-40ea-8f6b-27f947637ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:58.124749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6abc-a90b-40ea-8f6b-27f947637ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:58.125112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6abc-a90b-40ea-8f6b-27f947637ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:58.126152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da6abc-a90b-40ea-8f6b-27f947637ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.126284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da6abc-a90b-40ea-8f6b-27f947637ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:58.128297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da6abc-a90b-40ea-8f6b-27f947637ac5 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:58.148612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:58.154475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1124/2266] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:51:58.157346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d07afb-3f0e-4352-b09f-9d7496dc21b5 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:58.163855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:58.165540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:58.165540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:58.166013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] PATH_INFO: `/v3/projects/e217f3625a58433cabf6d750a17e8039/users/a979fd878af04915b18a1ed246dc32e0/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.168479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e217f3625a58433cabf6d750a17e8039, user_id=a979fd878af04915b18a1ed246dc32e0, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:58.171159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:58.183401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1143/2267] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:51:58 2022] PUT /identity/v3/projects/e217f3625a58433cabf6d750a17e8039/users/a979fd878af04915b18a1ed246dc32e0/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:58.193624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:58.194274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:58.194274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.229173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] MFA Rules not processed for user `a979fd878af04915b18a1ed246dc32e0`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:58.231756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:58.345239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1125/2268] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:58 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:58.359279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:58.361060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:58.423101 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:58.425119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:58.439144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400f6a4-d235-4bbc-bd45-286dfce73bd3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:58.443203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:58.443981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:58.444213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:58.444485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.456366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:58.458640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:58.471832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1144/2269] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:58 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:58.515862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:58.520162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4f455-bf59-48d9-9f91-6fb5404890a9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:58.525835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:58.527065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:58.527515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:58.527954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] PATH_INFO: `/v3/projects/27fd3f12712e4cf28dc4e13b0bc514ae` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.529636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27fd3f12712e4cf28dc4e13b0bc514ae)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:58.531585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:58.588075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27fd3f12712e4cf28dc4e13b0bc514ae', 'request_id': 'req-a4adee07-4da6-4f04-a239-d67a5ad20cb6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.597425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27fd3f12712e4cf28dc4e13b0bc514ae', 'request_id': 'req-a4adee07-4da6-4f04-a239-d67a5ad20cb6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.604719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27fd3f12712e4cf28dc4e13b0bc514ae', 'request_id': 'req-a4adee07-4da6-4f04-a239-d67a5ad20cb6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.616316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] The token cache is being invalidate because project 27fd3f12712e4cf28dc4e13b0bc514ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:58.616607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.617270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a4adee07-4da6-4f04-a239-d67a5ad20cb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.619056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1126/2270] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:58 2022] DELETE /identity/v3/projects/27fd3f12712e4cf28dc4e13b0bc514ae => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:58.626769 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:58.639898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d0794-920f-41cd-9940-86f497ea6c7c service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:58.763864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:58.768843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:58.771805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:58.780557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] PATH_INFO: `/v3/users/40138e6c4e7d4ccc8e25a9993b0877da` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.796679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40138e6c4e7d4ccc8e25a9993b0877da)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:58.800983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:58.839723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:58.886983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40138e6c4e7d4ccc8e25a9993b0877da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.897517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40138e6c4e7d4ccc8e25a9993b0877da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.909268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40138e6c4e7d4ccc8e25a9993b0877da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.911075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40138e6c4e7d4ccc8e25a9993b0877da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.911783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40138e6c4e7d4ccc8e25a9993b0877da'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:58.949203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1145/2271] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:58 2022] DELETE /identity/v3/users/40138e6c4e7d4ccc8e25a9993b0877da => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:51:58.951162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:58.951615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:58.951802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:58.977643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:51:58.979021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:59.073052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1127/2272] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:51:59.086917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:59.088585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:59.170002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3b92b-2075-4657-b65e-7dd1e5ee17a9 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:59.175158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:59.176177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:59.176535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:59.177206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:59.188016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:59.190171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:59.215579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1146/2273] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:51:59.654262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:59.655833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:51:59.668276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef11fe-0726-4594-ac09-4846993686b9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:59.673513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:59.674471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:59.674753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:59.675028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] PATH_INFO: `/v3/projects/03a5f3a40398462fa1636ff28ce278e8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:59.676257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a5f3a40398462fa1636ff28ce278e8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:59.678287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:59.729322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a5f3a40398462fa1636ff28ce278e8', 'request_id': 'req-59b16127-ce3c-40e4-ad0f-47be8f41e409'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.737549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a5f3a40398462fa1636ff28ce278e8', 'request_id': 'req-59b16127-ce3c-40e4-ad0f-47be8f41e409'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.744949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a5f3a40398462fa1636ff28ce278e8', 'request_id': 'req-59b16127-ce3c-40e4-ad0f-47be8f41e409'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.756466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] The token cache is being invalidate because project 03a5f3a40398462fa1636ff28ce278e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:51:59.756827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.757493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.759061 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1128/2274] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:51:59 2022] DELETE /identity/v3/projects/03a5f3a40398462fa1636ff28ce278e8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:51:59.763975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:51:59.775444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d3f9d-1988-4f8d-a3a3-e4323ba92ece service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:51:59.855768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:51:59.856850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:51:59.857434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:51:59.857823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] PATH_INFO: `/v3/users/a979fd878af04915b18a1ed246dc32e0` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:51:59.859140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a979fd878af04915b18a1ed246dc32e0)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:51:59.862109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:51:59.923204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:51:59.930101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a979fd878af04915b18a1ed246dc32e0'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.939366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a979fd878af04915b18a1ed246dc32e0'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.947874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a979fd878af04915b18a1ed246dc32e0'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.949080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a979fd878af04915b18a1ed246dc32e0'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.949861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-9a114f40-0e59-4a78-88f2-20b888ca7bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a979fd878af04915b18a1ed246dc32e0'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:51:59.970306 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1147/2275] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:51:59 2022] DELETE /identity/v3/users/a979fd878af04915b18a1ed246dc32e0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:00.281562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:00.285577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:00.337216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b16127-ce3c-40e4-ad0f-47be8f41e409 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:00.353120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:00.426595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:00.427489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.427923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.428234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] PATH_INFO: `/v3/projects/e217f3625a58433cabf6d750a17e8039` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.429698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e217f3625a58433cabf6d750a17e8039)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:00.431669 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:00.492846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e217f3625a58433cabf6d750a17e8039', 'request_id': 'req-047a26e2-039b-4481-a6e1-cea1b6914ecc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:00.504667 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e217f3625a58433cabf6d750a17e8039', 'request_id': 'req-047a26e2-039b-4481-a6e1-cea1b6914ecc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:00.514366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e217f3625a58433cabf6d750a17e8039', 'request_id': 'req-047a26e2-039b-4481-a6e1-cea1b6914ecc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:00.530479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] The token cache is being invalidate because project e217f3625a58433cabf6d750a17e8039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:52:00.530903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:00.531634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-047a26e2-039b-4481-a6e1-cea1b6914ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:00.536371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1129/2276] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:52:00 2022] DELETE /identity/v3/projects/e217f3625a58433cabf6d750a17e8039 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:00.540833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a6352-b695-4c0d-83ad-a07960c66a83 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.541834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a6352-b695-4c0d-83ad-a07960c66a83 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.542669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a6352-b695-4c0d-83ad-a07960c66a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.571557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-514a6352-b695-4c0d-83ad-a07960c66a83 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:00.572914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-514a6352-b695-4c0d-83ad-a07960c66a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:00.666918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1148/2277] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:00.671194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4cbfb-b93a-4b59-9aa2-c55682c5e591 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.671500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4cbfb-b93a-4b59-9aa2-c55682c5e591 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.671800 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4cbfb-b93a-4b59-9aa2-c55682c5e591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.695779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6ec4cbfb-b93a-4b59-9aa2-c55682c5e591 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:00.697041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6ec4cbfb-b93a-4b59-9aa2-c55682c5e591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:00.708732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1130/2278] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:00.712942 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a6352-b695-4c0d-83ad-a07960c66a83 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:00.725581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a6352-b695-4c0d-83ad-a07960c66a83 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:00.731802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:00.733163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.733414 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.733688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.734431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:52:00.734742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:00.736403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:00.749450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1149/2279] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:52:00.753825 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4cbfb-b93a-4b59-9aa2-c55682c5e591 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:00.764225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4cbfb-b93a-4b59-9aa2-c55682c5e591 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:00.770009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:00.770545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.770836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.771081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.771841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:00.774063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:00.818805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1131/2280] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:52:00 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:00.823192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:00.833471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c1cdc-5846-43a3-83d1-1ff49e0c980f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:00.892237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:00.892912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.893189 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.893457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.894166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:00.896123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:00.921373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1150/2281] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:52:00 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:00.922606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:00.935966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832e29-749a-41e3-8c1e-756236652b64 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:00.943108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:00.943907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.944327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.944783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.945534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:00.947164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:00.961323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1132/2282] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:00.966062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:00.976867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0ce12-7f79-4b0a-9434-f6711b1e3be5 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:00.984298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42af6f-faa3-477b-a1bc-93cc2b3c806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:00.985994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42af6f-faa3-477b-a1bc-93cc2b3c806e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:00.986427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42af6f-faa3-477b-a1bc-93cc2b3c806e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:00.986928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42af6f-faa3-477b-a1bc-93cc2b3c806e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:00.988100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42af6f-faa3-477b-a1bc-93cc2b3c806e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:00.991055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42af6f-faa3-477b-a1bc-93cc2b3c806e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.013117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1151/2283] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:01.014940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.029842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0555d3-16b1-43f5-a128-835b44e9414a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.034681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.035601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.035886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.036193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] PATH_INFO: `/v3/projects/e2a42faa543a44cf8403ae20e45eaeb3/users/462f1a6a8a0e4cc780505763c94fd2f1/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.052081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2a42faa543a44cf8403ae20e45eaeb3, user_id=462f1a6a8a0e4cc780505763c94fd2f1, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.055387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.072300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1133/2284] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:01 2022] PUT /identity/v3/projects/e2a42faa543a44cf8403ae20e45eaeb3/users/462f1a6a8a0e4cc780505763c94fd2f1/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:01.090540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989585e1-2ade-45c4-bacb-356ed3db490d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.090817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989585e1-2ade-45c4-bacb-356ed3db490d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.091099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989585e1-2ade-45c4-bacb-356ed3db490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.122144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-989585e1-2ade-45c4-bacb-356ed3db490d None None] MFA Rules not processed for user `462f1a6a8a0e4cc780505763c94fd2f1`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:01.123699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-989585e1-2ade-45c4-bacb-356ed3db490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:01.133360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:01.191452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1152/2285] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:01.196132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.242542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:01.267169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed058d48-9c9e-4f5d-8a08-41218e8ca6f4 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.273425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.274098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.274377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.274652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.275354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.277345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.305573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1134/2286] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:52:01 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:01.309722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989585e1-2ade-45c4-bacb-356ed3db490d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.320578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989585e1-2ade-45c4-bacb-356ed3db490d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.384678 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.385834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.385834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.385834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.386430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.389094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.415386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1153/2287] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:52:01 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:52:01.416638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.427589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7083e5-e21c-4b5d-9edd-77e8f5b0ce3a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.433104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.433806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.434082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.434354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.435086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.436765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.449021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1135/2288] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:52:01.453832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.464351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbade5-eb9c-4394-a675-5c3b51789820 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.469212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.470098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.470380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.470655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] PATH_INFO: `/v3/projects/3d2b0d1521cd4db5a947487e2854646e/users/9ceb77cfdbbb4fa1bce11da2aca0dca5/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.487138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2b0d1521cd4db5a947487e2854646e, user_id=9ceb77cfdbbb4fa1bce11da2aca0dca5, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.490178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.507708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1154/2289] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:01 2022] PUT /identity/v3/projects/3d2b0d1521cd4db5a947487e2854646e/users/9ceb77cfdbbb4fa1bce11da2aca0dca5/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:52:01.508745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.519221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85572de-2a4c-4b05-8d1a-042e065dc67d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.580727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bcf6b-3c3a-4df1-9d7f-a676ddd0086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.581409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bcf6b-3c3a-4df1-9d7f-a676ddd0086e admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.581685 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bcf6b-3c3a-4df1-9d7f-a676ddd0086e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.581958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bcf6b-3c3a-4df1-9d7f-a676ddd0086e admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.582717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bcf6b-3c3a-4df1-9d7f-a676ddd0086e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.584405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bcf6b-3c3a-4df1-9d7f-a676ddd0086e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.597857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1136/2290] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:52:01.602186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.613897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532d1e6-2c5f-4290-a52f-a9bd6e1de5cd admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.619161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.620145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.620717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.621015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] PATH_INFO: `/v3/projects/3d2b0d1521cd4db5a947487e2854646e/users/9ceb77cfdbbb4fa1bce11da2aca0dca5/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.623089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2b0d1521cd4db5a947487e2854646e, user_id=9ceb77cfdbbb4fa1bce11da2aca0dca5, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.625876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.637860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1155/2291] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:01 2022] PUT /identity/v3/projects/3d2b0d1521cd4db5a947487e2854646e/users/9ceb77cfdbbb4fa1bce11da2aca0dca5/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:01.649323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa430e-0024-4582-aead-2c48d2b414a9 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.649834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa430e-0024-4582-aead-2c48d2b414a9 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.650114 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa430e-0024-4582-aead-2c48d2b414a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.681645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9dfa430e-0024-4582-aead-2c48d2b414a9 None None] MFA Rules not processed for user `9ceb77cfdbbb4fa1bce11da2aca0dca5`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:01.683591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9dfa430e-0024-4582-aead-2c48d2b414a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:01.765127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1137/2292] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:01.776184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:01.777962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:01.863257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317d97a-a0bd-4ed4-8969-b6ca030f71bd admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:01.868681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e6c8a-8709-48c9-9dae-3d3bbd1e71fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:01.870040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6c8a-8709-48c9-9dae-3d3bbd1e71fb service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:01.870441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6c8a-8709-48c9-9dae-3d3bbd1e71fb service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:01.870799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6c8a-8709-48c9-9dae-3d3bbd1e71fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:01.885522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e6c8a-8709-48c9-9dae-3d3bbd1e71fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:01.888351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e6c8a-8709-48c9-9dae-3d3bbd1e71fb service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:01.902352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1156/2293] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:52:02.389807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa430e-0024-4582-aead-2c48d2b414a9 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:02.428465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:02.465040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa430e-0024-4582-aead-2c48d2b414a9 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:02.469656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:02.470537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:02.470819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:02.471093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] PATH_INFO: `/v3/users/462f1a6a8a0e4cc780505763c94fd2f1` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:02.472327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462f1a6a8a0e4cc780505763c94fd2f1)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:02.474252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:02.525889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462f1a6a8a0e4cc780505763c94fd2f1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:02.526610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462f1a6a8a0e4cc780505763c94fd2f1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:02.538879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462f1a6a8a0e4cc780505763c94fd2f1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:02.548316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462f1a6a8a0e4cc780505763c94fd2f1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:02.548957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462f1a6a8a0e4cc780505763c94fd2f1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:02.561749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1138/2294] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:52:02 2022] DELETE /identity/v3/users/462f1a6a8a0e4cc780505763c94fd2f1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:02.566193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:02.566972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:02.567348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:02.586238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:02.596539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:02.597896 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:02.661432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1157/2295] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:02.671301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:02.672782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:02.733429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9645-9396-4de0-bb73-e4b1af509af5 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:02.738360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:02.739406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:02.739738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:02.740121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:02.749972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:02.751463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:02.762428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1139/2296] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:52:03.069443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:03.072487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:03.109014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1c4d2-3b58-4c33-a4e0-0ad8f53542a9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:03.116542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:03.118467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:03.118848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:03.119569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] PATH_INFO: `/v3/projects/e2a42faa543a44cf8403ae20e45eaeb3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:03.121557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2a42faa543a44cf8403ae20e45eaeb3)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:03.124819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:03.186297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2a42faa543a44cf8403ae20e45eaeb3', 'request_id': 'req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.193600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2a42faa543a44cf8403ae20e45eaeb3', 'request_id': 'req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.202631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2a42faa543a44cf8403ae20e45eaeb3', 'request_id': 'req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.215197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] The token cache is being invalidate because project e2a42faa543a44cf8403ae20e45eaeb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:52:03.215605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.216334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.218385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1158/2297] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:52:03 2022] DELETE /identity/v3/projects/e2a42faa543a44cf8403ae20e45eaeb3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:03.222365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:03.238323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43696ebb-e65e-4874-8769-64abe1f30c80 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:03.314562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:03.323499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:03.324478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:03.324784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:03.325224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] PATH_INFO: `/v3/users/9ceb77cfdbbb4fa1bce11da2aca0dca5` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:03.326692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ceb77cfdbbb4fa1bce11da2aca0dca5)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:03.329653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:03.403722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ceb77cfdbbb4fa1bce11da2aca0dca5'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.404745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ceb77cfdbbb4fa1bce11da2aca0dca5'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.416943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ceb77cfdbbb4fa1bce11da2aca0dca5'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.425848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ceb77cfdbbb4fa1bce11da2aca0dca5'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.426468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-fb399748-4e78-4e2f-8c9a-8e8ca5ac5da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ceb77cfdbbb4fa1bce11da2aca0dca5'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.440141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1140/2298] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:52:03 2022] DELETE /identity/v3/users/9ceb77cfdbbb4fa1bce11da2aca0dca5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:03.731158 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:03.733085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:03.751052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9d9b5-96fb-4e7d-acfe-e60d2854fd97 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:03.809399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:03.810265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:03.810540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:03.810810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] PATH_INFO: `/v3/projects/3d2b0d1521cd4db5a947487e2854646e` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:03.812042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2b0d1521cd4db5a947487e2854646e)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:03.813937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:03.850105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:03.865619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2b0d1521cd4db5a947487e2854646e', 'request_id': 'req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.872390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2b0d1521cd4db5a947487e2854646e', 'request_id': 'req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.881519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2b0d1521cd4db5a947487e2854646e', 'request_id': 'req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.892603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] The token cache is being invalidate because project 3d2b0d1521cd4db5a947487e2854646e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:52:03.892896 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.893531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7ac33c4d-c868-4dd8-a9b3-b8c56d0c82de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:03.895083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1159/2299] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:52:03 2022] DELETE /identity/v3/projects/3d2b0d1521cd4db5a947487e2854646e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:03.900408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402d0e-ca28-4e8b-8cdc-94adae92221e None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:03.900697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402d0e-ca28-4e8b-8cdc-94adae92221e None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:03.900973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402d0e-ca28-4e8b-8cdc-94adae92221e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:03.920715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-68402d0e-ca28-4e8b-8cdc-94adae92221e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:03.921634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-68402d0e-ca28-4e8b-8cdc-94adae92221e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:03.984547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1141/2300] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:52:03.996477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5b7f-b1dd-4119-b770-594c59f4a6c0 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:03.996785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5b7f-b1dd-4119-b770-594c59f4a6c0 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:03.997056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5b7f-b1dd-4119-b770-594c59f4a6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.016285 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e12f5b7f-b1dd-4119-b770-594c59f4a6c0 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:04.017908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e12f5b7f-b1dd-4119-b770-594c59f4a6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:04.030610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1160/2301] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:04.035088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402d0e-ca28-4e8b-8cdc-94adae92221e None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:04.045762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402d0e-ca28-4e8b-8cdc-94adae92221e None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:04.051980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:04.052656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.052936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.053211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.053939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:52:04.054254 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:04.055855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:04.068842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1142/2302] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:52:04.073717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5b7f-b1dd-4119-b770-594c59f4a6c0 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:04.087671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5b7f-b1dd-4119-b770-594c59f4a6c0 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:04.094314 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:04.094959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.095237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.095504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.096233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:04.098136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:04.128230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1161/2303] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:52:04 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:04.132631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:04.143463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe6c1d-25cc-411b-9f42-042a2d1e779f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:04.206917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:04.207592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.207874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.208180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.209257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:04.211177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:04.219621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:04.236954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1143/2304] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:52:04 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:04.241103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:04.252118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b20e8-20be-4272-9b39-be02dc3bc3b3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:04.257162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:04.257822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.258105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.258379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.259144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:04.260909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:04.273785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1162/2305] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:04.277917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:04.289539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532f553-4f29-4cb0-b4b5-fb308da31c5b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:04.294565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241fa33-a83b-4664-b621-fd32b6e24a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:04.295238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241fa33-a83b-4664-b621-fd32b6e24a73 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.295664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241fa33-a83b-4664-b621-fd32b6e24a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.295943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241fa33-a83b-4664-b621-fd32b6e24a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.296725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241fa33-a83b-4664-b621-fd32b6e24a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:04.298452 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241fa33-a83b-4664-b621-fd32b6e24a73 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:04.309015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1144/2306] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:04.313310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:04.323731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac143c-ea78-42d0-932d-3390544f0524 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:04.328545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:04.329432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.329709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.329976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] PATH_INFO: `/v3/projects/1867c3462bcf42b59094717b7d36d4f1/users/12ff5e74c0dd47baaa7d4e6cbeaf79db/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.345269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1867c3462bcf42b59094717b7d36d4f1, user_id=12ff5e74c0dd47baaa7d4e6cbeaf79db, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:04.348036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:04.361837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1163/2307] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:04 2022] PUT /identity/v3/projects/1867c3462bcf42b59094717b7d36d4f1/users/12ff5e74c0dd47baaa7d4e6cbeaf79db/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:04.373259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7a48e-5df7-4eb3-b757-0ad2d8433242 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.373639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7a48e-5df7-4eb3-b757-0ad2d8433242 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.373993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7a48e-5df7-4eb3-b757-0ad2d8433242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.406091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ccd7a48e-5df7-4eb3-b757-0ad2d8433242 None None] MFA Rules not processed for user `12ff5e74c0dd47baaa7d4e6cbeaf79db`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:04.407427 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ccd7a48e-5df7-4eb3-b757-0ad2d8433242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:04.477923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1145/2308] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:04.490825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:04.493263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:04.568907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da1936-c887-47d0-ab90-ee22b465fe68 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:04.574007 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0471174-6fbe-41ee-8c62-988d282cfefb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:04.575011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471174-6fbe-41ee-8c62-988d282cfefb service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:04.575298 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471174-6fbe-41ee-8c62-988d282cfefb service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:04.575621 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0471174-6fbe-41ee-8c62-988d282cfefb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:04.586129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0471174-6fbe-41ee-8c62-988d282cfefb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:04.587596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0471174-6fbe-41ee-8c62-988d282cfefb service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:04.597864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1164/2309] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:52:14.474745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:14.593274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:27.750906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7a48e-5df7-4eb3-b757-0ad2d8433242 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:27.815911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7a48e-5df7-4eb3-b757-0ad2d8433242 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:27.827342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:27.828121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:27.828284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:27.828567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] PATH_INFO: `/v3/users/12ff5e74c0dd47baaa7d4e6cbeaf79db` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:27.830160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12ff5e74c0dd47baaa7d4e6cbeaf79db)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:27.832461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:27.881190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12ff5e74c0dd47baaa7d4e6cbeaf79db'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:27.881835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12ff5e74c0dd47baaa7d4e6cbeaf79db'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:27.893975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12ff5e74c0dd47baaa7d4e6cbeaf79db'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:27.904124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12ff5e74c0dd47baaa7d4e6cbeaf79db'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:27.904764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12ff5e74c0dd47baaa7d4e6cbeaf79db'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:27.918165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1146/2310] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:52:27 2022] DELETE /identity/v3/users/12ff5e74c0dd47baaa7d4e6cbeaf79db => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:27.922686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:27.923212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:27.923212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:27.941870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:27.943080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:28.004322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1165/2311] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:28.014148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.015730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:28.077607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a748fdb-ab1a-456e-b708-393c6abafdf2 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.081406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c906a-cb94-4821-8a9d-4e807131dc81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.082538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c906a-cb94-4821-8a9d-4e807131dc81 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.082538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c906a-cb94-4821-8a9d-4e807131dc81 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.082741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c906a-cb94-4821-8a9d-4e807131dc81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.096030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c906a-cb94-4821-8a9d-4e807131dc81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.096030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c906a-cb94-4821-8a9d-4e807131dc81 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.106529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1147/2312] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:52:28.386434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.387953 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:28.400198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e3df2-316a-458e-ac5d-c07942d1e967 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.405081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.405939 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.406222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.406493 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] PATH_INFO: `/v3/projects/1867c3462bcf42b59094717b7d36d4f1` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.408510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1867c3462bcf42b59094717b7d36d4f1)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.410552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.462561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1867c3462bcf42b59094717b7d36d4f1', 'request_id': 'req-ea7908da-239b-4d12-8aca-3e7122230220'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:28.468960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1867c3462bcf42b59094717b7d36d4f1', 'request_id': 'req-ea7908da-239b-4d12-8aca-3e7122230220'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:28.479221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1867c3462bcf42b59094717b7d36d4f1', 'request_id': 'req-ea7908da-239b-4d12-8aca-3e7122230220'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:28.489551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] The token cache is being invalidate because project 1867c3462bcf42b59094717b7d36d4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:52:28.489834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:28.490500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ea7908da-239b-4d12-8aca-3e7122230220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:28.492209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1166/2313] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:52:28 2022] DELETE /identity/v3/projects/1867c3462bcf42b59094717b7d36d4f1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:28.497284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1de9-dd79-4fca-ba4c-60b4089bbba5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.497573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1de9-dd79-4fca-ba4c-60b4089bbba5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.497850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c1de9-dd79-4fca-ba4c-60b4089bbba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.517016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7e4c1de9-dd79-4fca-ba4c-60b4089bbba5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:28.518251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7e4c1de9-dd79-4fca-ba4c-60b4089bbba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:28.537071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:28.581518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1148/2314] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:28.590469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b3f51-e2f0-4beb-80a2-e278b06bd7ae None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.590758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b3f51-e2f0-4beb-80a2-e278b06bd7ae None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.591038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b3f51-e2f0-4beb-80a2-e278b06bd7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.611511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9b1b3f51-e2f0-4beb-80a2-e278b06bd7ae None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:28.613118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9b1b3f51-e2f0-4beb-80a2-e278b06bd7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:28.622235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1167/2315] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:52:28.626712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1de9-dd79-4fca-ba4c-60b4089bbba5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.636618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c1de9-dd79-4fca-ba4c-60b4089bbba5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.641373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.642030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.642362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.642592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.643325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:52:28.643640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.645288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.659258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1149/2316] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:52:28.663652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b3f51-e2f0-4beb-80a2-e278b06bd7ae None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.673657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b3f51-e2f0-4beb-80a2-e278b06bd7ae None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.678552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.679216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.679495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.679824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.680534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.682454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.711751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1168/2317] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:52:28 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:28.716164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.726104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e8339-7745-4698-ab55-fd1095bbe945 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.784596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.785270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.785550 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.785826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.786536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.788525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.810448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1150/2318] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:52:28 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:28.814639 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.825087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacb0c5-5198-4c95-9409-e246ebfd841b admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.829900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.830553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.830891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.831180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.832196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.834310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.850156 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1169/2319] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:28.851369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.865122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0601f1-f5a2-4d02-915a-1ad8346155db admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.873654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e67aea-49be-41c1-862c-577d2d0ecbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.874386 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67aea-49be-41c1-862c-577d2d0ecbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.874668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67aea-49be-41c1-862c-577d2d0ecbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.874946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67aea-49be-41c1-862c-577d2d0ecbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.875692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e67aea-49be-41c1-862c-577d2d0ecbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.877360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e67aea-49be-41c1-862c-577d2d0ecbd6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.888820 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1151/2320] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:28.893173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:28.905046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f040bfd-0cef-482b-8cde-2dfe6f6fe30e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:28.910607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:28.914006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.914006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.914006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] PATH_INFO: `/v3/projects/93cde641200944a38b1958a2ac308994/users/014f9adeb36342c1a5ec0ffe9411eca6/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.929955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93cde641200944a38b1958a2ac308994, user_id=014f9adeb36342c1a5ec0ffe9411eca6, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:28.932664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:28.946802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1170/2321] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:28 2022] PUT /identity/v3/projects/93cde641200944a38b1958a2ac308994/users/014f9adeb36342c1a5ec0ffe9411eca6/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:28.958026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bd9da-a899-4c8c-b820-3e7cc5cc51bb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:28.958323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bd9da-a899-4c8c-b820-3e7cc5cc51bb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:28.958765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bd9da-a899-4c8c-b820-3e7cc5cc51bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:28.987032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f24bd9da-a899-4c8c-b820-3e7cc5cc51bb None None] MFA Rules not processed for user `014f9adeb36342c1a5ec0ffe9411eca6`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:28.988879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f24bd9da-a899-4c8c-b820-3e7cc5cc51bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:29.067443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1152/2322] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:29.080514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:29.083039 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:29.111029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:29.150617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef82b2-ab77-4755-bc4c-fd952f7d5a60 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:29.154858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:29.155548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:29.155831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:29.156165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:29.164838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:29.166575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:29.176525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1171/2323] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:52:34.555470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7e70-af0f-41d6-836d-c13a2ef574b7 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:34.555957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7e70-af0f-41d6-836d-c13a2ef574b7 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:34.556096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a7e70-af0f-41d6-836d-c13a2ef574b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:34.574543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1e6a7e70-af0f-41d6-836d-c13a2ef574b7 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:34.575706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1e6a7e70-af0f-41d6-836d-c13a2ef574b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:34.588292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:34.635576 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1153/2324] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:34.681392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:34.683145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:34.764874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dafad0-bbab-4a7f-afa0-d5dfb3ef3525 service glance] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:34.771081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:34.771735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:34.772019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:34.772312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:34.781234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:34.782845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:34.791134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1172/2325] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:52:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:52:35.396898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250a6b-0131-45b8-bfa2-53058196fe53 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:35.397187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250a6b-0131-45b8-bfa2-53058196fe53 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:35.397463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7250a6b-0131-45b8-bfa2-53058196fe53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:35.420030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b7250a6b-0131-45b8-bfa2-53058196fe53 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:35.421285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b7250a6b-0131-45b8-bfa2-53058196fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:35.429544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1154/2326] 10.222.0.50 () {58 vars in 967 bytes} [Thu Jun 30 15:52:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:35.443612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:35.454540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7f828-d29a-49bc-9246-d69a75d76ca8 service swift] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:35.459705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:35.460365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:35.460650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:35.460925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:35.469385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:35.471199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:35.479279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1173/2327] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:52:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:52:35.693075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d5c5a-5001-42e0-8963-676f3e425fba None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:35.693563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d5c5a-5001-42e0-8963-676f3e425fba None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:35.693738 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d5c5a-5001-42e0-8963-676f3e425fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:35.711091 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1a8d5c5a-5001-42e0-8963-676f3e425fba None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:35.712297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1a8d5c5a-5001-42e0-8963-676f3e425fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:35.719467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1155/2328] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:52:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:35.731355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:35.743367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1c04c-cbb5-4244-add7-2c5892985e62 service swift] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:35.748944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:35.749601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:35.749880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:35.750150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:35.760160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:35.762869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:35.776386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1174/2329] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:52:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:52:36.129567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2abb6-4c77-43c2-b1ad-dd861d94c4b6 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:36.129882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2abb6-4c77-43c2-b1ad-dd861d94c4b6 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:36.130183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2abb6-4c77-43c2-b1ad-dd861d94c4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:36.149407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-89e2abb6-4c77-43c2-b1ad-dd861d94c4b6 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:36.150607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-89e2abb6-4c77-43c2-b1ad-dd861d94c4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:36.159221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1156/2330] 10.222.0.50 () {58 vars in 967 bytes} [Thu Jun 30 15:52:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:36.171465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:36.183584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd54b30-e64a-4bcb-9ffb-a2347d38b62f service swift] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:36.189873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:36.190512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:36.190789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:36.191067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:36.199687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:36.201684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:36.210365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1175/2331] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:52:40.097126 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:40.097674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:40.097674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:40.117434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:40.118910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:40.128326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1157/2332] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:52:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:40.146941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:40.159637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782e3da-5025-4c4a-9337-db3fb3d26a19 service swift] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:40.167013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f948e1e-ba1b-4486-86d1-1667f65b46c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:40.168753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f948e1e-ba1b-4486-86d1-1667f65b46c1 service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:40.169309 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f948e1e-ba1b-4486-86d1-1667f65b46c1 service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:40.169744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f948e1e-ba1b-4486-86d1-1667f65b46c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:40.191796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f948e1e-ba1b-4486-86d1-1667f65b46c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:40.193627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f948e1e-ba1b-4486-86d1-1667f65b46c1 service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:40.202901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1176/2333] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:52:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:52:40.413397 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:40.415423 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:40.505231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3708644a-e816-4c9a-9cc2-e88492e5f594 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:40.510337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:40.511283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:40.511564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:40.511841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] PATH_INFO: `/v3/users/f2b4682563a24786804dee37d5ee31eb` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:40.517213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b4682563a24786804dee37d5ee31eb)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:40.519215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:40.576261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b4682563a24786804dee37d5ee31eb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:40.576841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b4682563a24786804dee37d5ee31eb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:40.590710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b4682563a24786804dee37d5ee31eb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:40.598449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b4682563a24786804dee37d5ee31eb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:40.599132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b4682563a24786804dee37d5ee31eb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:40.612865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1158/2334] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:52:40 2022] DELETE /identity/v3/users/f2b4682563a24786804dee37d5ee31eb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:40.618069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:40.618637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:40.618917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:40.637766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:40.638987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:40.708763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:40.724028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1177/2335] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:40.730725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:40.732219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:40.753718 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:40.800690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fadbea-e01c-4cc4-8744-f50d4ba8b57f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:40.804756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed63be-631e-43eb-9d46-0181d6264765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:40.805413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed63be-631e-43eb-9d46-0181d6264765 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:40.805699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed63be-631e-43eb-9d46-0181d6264765 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:40.805975 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed63be-631e-43eb-9d46-0181d6264765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:40.815600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed63be-631e-43eb-9d46-0181d6264765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:40.817081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed63be-631e-43eb-9d46-0181d6264765 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:40.830013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1159/2336] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:52:41.018753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.020499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:41.048747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9b8d8-75d1-49be-b3cf-2313ddb6788d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.056215 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.057165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.057561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.057849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] PATH_INFO: `/v3/projects/5ac1871137494335b9cee22170d3315b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.059142 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac1871137494335b9cee22170d3315b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.061240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:41.115381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac1871137494335b9cee22170d3315b', 'request_id': 'req-a0787050-881a-4ebd-b508-2b32a314858e'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:41.122668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac1871137494335b9cee22170d3315b', 'request_id': 'req-a0787050-881a-4ebd-b508-2b32a314858e'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:41.132224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac1871137494335b9cee22170d3315b', 'request_id': 'req-a0787050-881a-4ebd-b508-2b32a314858e'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:41.144413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] The token cache is being invalidate because project 5ac1871137494335b9cee22170d3315b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:52:41.145032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:41.145918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a0787050-881a-4ebd-b508-2b32a314858e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:41.147477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1178/2337] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:52:41 2022] DELETE /identity/v3/projects/5ac1871137494335b9cee22170d3315b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:41.154903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa54fa-c8b7-433a-b643-fd7dc0f245e8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.155416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa54fa-c8b7-433a-b643-fd7dc0f245e8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.155648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa54fa-c8b7-433a-b643-fd7dc0f245e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.174873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-68fa54fa-c8b7-433a-b643-fd7dc0f245e8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:41.176270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-68fa54fa-c8b7-433a-b643-fd7dc0f245e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:41.257672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1160/2338] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:41.323228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eb128-935d-4a65-bdb6-71859d92d28e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.323518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eb128-935d-4a65-bdb6-71859d92d28e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.323914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eb128-935d-4a65-bdb6-71859d92d28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.344337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-478eb128-935d-4a65-bdb6-71859d92d28e None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:41.345470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-478eb128-935d-4a65-bdb6-71859d92d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:41.353567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1179/2339] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:41.358475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa54fa-c8b7-433a-b643-fd7dc0f245e8 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.370027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa54fa-c8b7-433a-b643-fd7dc0f245e8 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.374828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.375475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.375785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.376086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.376879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:52:41.377182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.378826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:41.392164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1161/2340] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:52:41.398344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eb128-935d-4a65-bdb6-71859d92d28e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.416336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eb128-935d-4a65-bdb6-71859d92d28e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.425412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.426101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.426567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.426999 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.428029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.430519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:41.464668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1180/2341] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:52:41 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:41.469096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.484525 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f46738-9d80-44c3-9a20-f28e657d7645 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.635169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.636063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.636317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.636598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.637306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.639250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:41.667628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1162/2342] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:52:41 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:52:41.668966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.681770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e8da8-8ca2-479d-98cd-0b0177ecfaa8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.688452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.689175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.689841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.689841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.690490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.692136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:41.704246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1181/2343] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:41.708567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.721130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3ccfc-787e-44c8-a435-6cc04d31edc6 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.729344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40273c26-7c88-48a5-833b-c6da34158fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.730162 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273c26-7c88-48a5-833b-c6da34158fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.730539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273c26-7c88-48a5-833b-c6da34158fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.730932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40273c26-7c88-48a5-833b-c6da34158fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.731668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40273c26-7c88-48a5-833b-c6da34158fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.733824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40273c26-7c88-48a5-833b-c6da34158fd4 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:41.752593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1163/2344] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:41.753429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.767548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d46ed-7120-44ed-8ff3-61ea6eb445a6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.772812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.773671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.774045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.774404 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] PATH_INFO: `/v3/projects/affc1e688df04803bb823ccadf904c0a/users/2de768b55f9a495295113de2ce1279e5/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.789616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] RBAC: Authorizing `identity:create_grant(project_id=affc1e688df04803bb823ccadf904c0a, user_id=2de768b55f9a495295113de2ce1279e5, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.792360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:41.805980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1182/2345] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:41 2022] PUT /identity/v3/projects/affc1e688df04803bb823ccadf904c0a/users/2de768b55f9a495295113de2ce1279e5/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:52:41.811231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b05c0-221b-483f-bef8-5f1d94b4fb0f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.811522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b05c0-221b-483f-bef8-5f1d94b4fb0f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.811804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b05c0-221b-483f-bef8-5f1d94b4fb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.829522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6f5b05c0-221b-483f-bef8-5f1d94b4fb0f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:41.830724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6f5b05c0-221b-483f-bef8-5f1d94b4fb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:41.849503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:41.903570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1164/2346] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:41.914766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:41.916891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:41.977058 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da317f-e8fc-41e7-91c4-43a16469915c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:41.981359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:41.982577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:41.983022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:41.983675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:41.995668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:41.997337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:42.009566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1183/2347] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:52:48.251489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129330ae-a1cb-4845-ad27-31882dbad34d None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:48.252639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129330ae-a1cb-4845-ad27-31882dbad34d None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:48.254211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129330ae-a1cb-4845-ad27-31882dbad34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:48.352635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-129330ae-a1cb-4845-ad27-31882dbad34d None None] MFA Rules not processed for user `2de768b55f9a495295113de2ce1279e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:48.355285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-129330ae-a1cb-4845-ad27-31882dbad34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:48.521864 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1165/2348] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:48.526216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:48.554611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600b4da-389d-475b-8c85-1e6f47be7b8c service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:48.560753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:48.561672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:48.562117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:48.572525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:48.573387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:48.575375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:48.602473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:48.613329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1184/2349] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:52:48 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:52:48.617120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129330ae-a1cb-4845-ad27-31882dbad34d None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:48.633179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129330ae-a1cb-4845-ad27-31882dbad34d None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:48.727681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:48.728666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:48.729049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:48.729862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:48.730725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:48.733613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:48.775554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1166/2350] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:52:48 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:52:48.778528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:48.790880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad060b2f-1e5d-4f19-bdb9-9eff6c378432 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:48.797808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:48.798865 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:48.799427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:48.799915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:48.801352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:48.803275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:48.816948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1185/2351] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:48.821698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:48.838497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1882f-ee21-4e21-bb23-a7507b522be5 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:48.844832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:48.845769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:48.846119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:48.846320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] PATH_INFO: `/v3/projects/98deee3253214389826d8633dfb172cc/users/301f0098c8ce48f2914d5e7ec2a4bc5a/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:48.869489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98deee3253214389826d8633dfb172cc, user_id=301f0098c8ce48f2914d5e7ec2a4bc5a, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:48.873101 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:48.904292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1167/2352] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:48 2022] PUT /identity/v3/projects/98deee3253214389826d8633dfb172cc/users/301f0098c8ce48f2914d5e7ec2a4bc5a/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:48.908920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:48.925644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c8891-48f4-4bc0-8b98-0d6c4a93c0c1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:49.053560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:49.054366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:49.054670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:49.054930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:49.055699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:49.057359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:49.071281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1186/2353] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:52:49.075366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:49.087938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e03abf-8aa5-45e9-aac4-fb9f2c39cf60 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:49.095670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afca343-e269-496e-93eb-20acc5701550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:49.096796 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca343-e269-496e-93eb-20acc5701550 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:49.097181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca343-e269-496e-93eb-20acc5701550 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:49.097554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afca343-e269-496e-93eb-20acc5701550 admin admin] PATH_INFO: `/v3/projects/98deee3253214389826d8633dfb172cc/users/301f0098c8ce48f2914d5e7ec2a4bc5a/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:49.100165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afca343-e269-496e-93eb-20acc5701550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98deee3253214389826d8633dfb172cc, user_id=301f0098c8ce48f2914d5e7ec2a4bc5a, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:49.103431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afca343-e269-496e-93eb-20acc5701550 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:49.116314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1168/2354] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:52:49 2022] PUT /identity/v3/projects/98deee3253214389826d8633dfb172cc/users/301f0098c8ce48f2914d5e7ec2a4bc5a/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:50.001653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:50.015829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:50.053282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82029ef-20e2-431a-9dfe-cdc2ffe4b1cf admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:50.254527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:50.255596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:50.255746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:50.256016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] PATH_INFO: `/v3/users/9d5e16507d3b490995eed013ca606554` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:50.264649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5e16507d3b490995eed013ca606554)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:50.266654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:50.353922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5e16507d3b490995eed013ca606554'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:50.371446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5e16507d3b490995eed013ca606554'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:50.392632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5e16507d3b490995eed013ca606554'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:50.395144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5e16507d3b490995eed013ca606554'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:50.396277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5e16507d3b490995eed013ca606554'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:50.467154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1187/2355] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:52:50 2022] DELETE /identity/v3/users/9d5e16507d3b490995eed013ca606554 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:50.471643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:50.472523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:50.473041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:50.511098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:50.526387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:50.529166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:50.645430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1169/2356] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:50.656529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:50.658777 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:50.666888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:50.815017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15d8a9-f3f9-4641-94b7-279e3b67d117 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:50.820930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57124ccc-66e0-425a-8834-47ea73e4e5c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:50.821764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57124ccc-66e0-425a-8834-47ea73e4e5c2 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:50.822723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57124ccc-66e0-425a-8834-47ea73e4e5c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:50.823101 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57124ccc-66e0-425a-8834-47ea73e4e5c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:50.835348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57124ccc-66e0-425a-8834-47ea73e4e5c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:50.837065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57124ccc-66e0-425a-8834-47ea73e4e5c2 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:50.863162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1188/2357] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:52:51.790898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:51.792920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:51.829690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636679c-b222-4355-9cd8-7c42e078eff5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:51.836845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:51.837937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:51.838372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:51.838859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] PATH_INFO: `/v3/projects/6a0d5c681e2449999e5c5829ff38945a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:51.840924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a0d5c681e2449999e5c5829ff38945a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:51.843171 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:51.997972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a0d5c681e2449999e5c5829ff38945a', 'request_id': 'req-a6de949f-a5a9-4715-beda-12f07232bbd2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:52.019042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a0d5c681e2449999e5c5829ff38945a', 'request_id': 'req-a6de949f-a5a9-4715-beda-12f07232bbd2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:52.055447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a0d5c681e2449999e5c5829ff38945a', 'request_id': 'req-a6de949f-a5a9-4715-beda-12f07232bbd2'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:52.084040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] The token cache is being invalidate because project 6a0d5c681e2449999e5c5829ff38945a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:52:52.084587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:52.088746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:52:52.118135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1170/2358] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:52:51 2022] DELETE /identity/v3/projects/6a0d5c681e2449999e5c5829ff38945a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:52:57.257643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafe93f-4ee1-42f9-a0cd-72c3f3ab8513 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:57.258382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafe93f-4ee1-42f9-a0cd-72c3f3ab8513 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:57.261654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafe93f-4ee1-42f9-a0cd-72c3f3ab8513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:57.352428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6dafe93f-4ee1-42f9-a0cd-72c3f3ab8513 None None] MFA Rules not processed for user `301f0098c8ce48f2914d5e7ec2a4bc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:52:57.353679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6dafe93f-4ee1-42f9-a0cd-72c3f3ab8513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:57.451239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1189/2359] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:52:57 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:52:57.657946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:52:57.661407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:57.766831 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de949f-a5a9-4715-beda-12f07232bbd2 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:52:57.778174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:52:57.778849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:52:57.779222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:52:57.779492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:52:57.781740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:52:57.850529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:52:57.964851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:52:57.969272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:52:57.985149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1171/2360] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:52:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:07.446667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=58 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:07.977121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=15 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:09.737247 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b134fb-8fe0-4166-9147-99bd4895912a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:09.737945 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b134fb-8fe0-4166-9147-99bd4895912a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:09.738055 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b134fb-8fe0-4166-9147-99bd4895912a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:09.790564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-81b134fb-8fe0-4166-9147-99bd4895912a None None] MFA Rules not processed for user `d30ffdb0374c4600a09476cfb3a07cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:09.791833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-81b134fb-8fe0-4166-9147-99bd4895912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:09.899783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1190/2361] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:10.723268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:10.725106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:10.895875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153579d-c441-4422-8a93-73533dfceefe service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:10.900788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:10.901671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:10.901951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:10.902224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] PATH_INFO: `/v3/users/014f9adeb36342c1a5ec0ffe9411eca6` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:10.903468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=014f9adeb36342c1a5ec0ffe9411eca6)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:10.905544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:10.987218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014f9adeb36342c1a5ec0ffe9411eca6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:10.987907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014f9adeb36342c1a5ec0ffe9411eca6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.003378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014f9adeb36342c1a5ec0ffe9411eca6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.013212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014f9adeb36342c1a5ec0ffe9411eca6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.013922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014f9adeb36342c1a5ec0ffe9411eca6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.036846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1172/2362] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:53:10 2022] DELETE /identity/v3/users/014f9adeb36342c1a5ec0ffe9411eca6 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:53:11.041129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6bd62-bc80-42e0-8452-98500904bf15 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:11.041611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6bd62-bc80-42e0-8452-98500904bf15 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:11.041709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa6bd62-bc80-42e0-8452-98500904bf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:11.079044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7aa6bd62-bc80-42e0-8452-98500904bf15 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:11.080281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7aa6bd62-bc80-42e0-8452-98500904bf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:11.148959 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1191/2363] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:11.158928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:11.160430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:11.187245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:11.275065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64596f-fb06-4b91-9855-86793a49e1b9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:11.281167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:11.281167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:11.281167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:11.281167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:11.290658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:11.292242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:11.310159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1173/2364] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:53:11.397059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:11.397370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:11.397715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:11.416855 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:11.418077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:11.484210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1192/2365] 10.222.0.50 () {58 vars in 967 bytes} [Thu Jun 30 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:11.497089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:11.499111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:11.584967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f4b83-4581-4767-b3d6-0d2ac5631b85 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:11.591900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60115de3-40b8-4fd9-9de3-e4f0c223e5ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:11.592785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60115de3-40b8-4fd9-9de3-e4f0c223e5ac service swift] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:11.593236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60115de3-40b8-4fd9-9de3-e4f0c223e5ac service swift] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:11.593619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60115de3-40b8-4fd9-9de3-e4f0c223e5ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:11.603802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60115de3-40b8-4fd9-9de3-e4f0c223e5ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:11.605593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60115de3-40b8-4fd9-9de3-e4f0c223e5ac service swift] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:11.615744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1174/2366] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:53:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:53:11.660995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:11.662885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:11.680130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11798e3-0f90-444f-aa23-a02589b358b4 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:11.691600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:11.692636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:11.692934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:11.693319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] PATH_INFO: `/v3/projects/93cde641200944a38b1958a2ac308994` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:11.694588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=93cde641200944a38b1958a2ac308994)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:11.696973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:11.710698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:11.752144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93cde641200944a38b1958a2ac308994', 'request_id': 'req-15c92931-990c-4fdc-8994-cb398be39d0c'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.758674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93cde641200944a38b1958a2ac308994', 'request_id': 'req-15c92931-990c-4fdc-8994-cb398be39d0c'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.768752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93cde641200944a38b1958a2ac308994', 'request_id': 'req-15c92931-990c-4fdc-8994-cb398be39d0c'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.783756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] The token cache is being invalidate because project 93cde641200944a38b1958a2ac308994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:53:11.784087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.785957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-15c92931-990c-4fdc-8994-cb398be39d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:11.787560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1193/2367] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:53:11 2022] DELETE /identity/v3/projects/93cde641200944a38b1958a2ac308994 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:53:11.792512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f7f35-991a-4517-8cfa-cf6199cf7bc2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:11.793002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f7f35-991a-4517-8cfa-cf6199cf7bc2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:11.793118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f7f35-991a-4517-8cfa-cf6199cf7bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:11.836344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-ed7f7f35-991a-4517-8cfa-cf6199cf7bc2 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:11.837860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ed7f7f35-991a-4517-8cfa-cf6199cf7bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:11.913409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:11.933120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1175/2368] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:11.942693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa81dd-a2b0-45c9-a5ba-c0758285f284 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:11.942984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa81dd-a2b0-45c9-a5ba-c0758285f284 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:11.943258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa81dd-a2b0-45c9-a5ba-c0758285f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:11.977131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-82fa81dd-a2b0-45c9-a5ba-c0758285f284 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:11.978403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-82fa81dd-a2b0-45c9-a5ba-c0758285f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:11.993383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1194/2369] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:11.999699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f7f35-991a-4517-8cfa-cf6199cf7bc2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:12.010354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f7f35-991a-4517-8cfa-cf6199cf7bc2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:12.021948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:12.022722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.022940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.023231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.023957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:53:12.024301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:12.025932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:12.045505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1176/2370] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:53:12.049728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa81dd-a2b0-45c9-a5ba-c0758285f284 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:12.061455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa81dd-a2b0-45c9-a5ba-c0758285f284 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:12.068895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:12.069969 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.070457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.070881 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.071736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:12.074377 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:12.124508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1195/2371] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:53:12 2022] POST /identity/v3/projects => generated 381 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:53:12.125724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:12.151522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c09c9e-9296-4912-8441-b8ea7026f41c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:12.257492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:12.258298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.258583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.258859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.259607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:12.261675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:12.293755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1177/2372] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:53:12 2022] POST /identity/v3/users => generated 291 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:53:12.297813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:12.313430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf370f4-f4c1-474c-9053-169c8ff42915 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:12.318463 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:12.319255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.320054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.320384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.321275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:12.323029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:12.335074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1196/2373] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:53:12.339358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:12.350032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6bc69-5cf8-4df8-9dcf-902aeb958c0e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:12.354903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e5095-e2a4-4318-aa3e-d6b100cd1646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:12.355671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e5095-e2a4-4318-aa3e-d6b100cd1646 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.356006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e5095-e2a4-4318-aa3e-d6b100cd1646 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.356601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e5095-e2a4-4318-aa3e-d6b100cd1646 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.357167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e5095-e2a4-4318-aa3e-d6b100cd1646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:12.358881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e5095-e2a4-4318-aa3e-d6b100cd1646 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:12.370917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1178/2374] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:53:12.375236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:12.385874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45282fa-59e7-4bd1-90c0-ca5a6f6142a9 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:12.390720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:12.391652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.391927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.392239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] PATH_INFO: `/v3/projects/cf17d9690bf54b98815f00334a3470bc/users/b9a7593adca849d49a95cec18d31ee14/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.408232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf17d9690bf54b98815f00334a3470bc, user_id=b9a7593adca849d49a95cec18d31ee14, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:12.411097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:12.431192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1197/2375] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:53:12 2022] PUT /identity/v3/projects/cf17d9690bf54b98815f00334a3470bc/users/b9a7593adca849d49a95cec18d31ee14/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:53:12.436424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd31acd-ce77-4850-a182-abfd189dafaa None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.436712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd31acd-ce77-4850-a182-abfd189dafaa None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.437032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd31acd-ce77-4850-a182-abfd189dafaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.455579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4bd31acd-ce77-4850-a182-abfd189dafaa None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:12.456830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4bd31acd-ce77-4850-a182-abfd189dafaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:12.523980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1179/2376] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:53:12.531842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:12.533594 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:12.600710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611601-bd77-429e-b1ec-9c779a4ae8d7 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:12.604521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:12.605293 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:12.605669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:12.606129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:12.617797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:12.619509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:12.630349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1198/2377] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:13.372413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fe70a-e8be-4420-ada5-c78455ff90b5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:13.372908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fe70a-e8be-4420-ada5-c78455ff90b5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:13.374931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2fe70a-e8be-4420-ada5-c78455ff90b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:13.413252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7b2fe70a-e8be-4420-ada5-c78455ff90b5 None None] MFA Rules not processed for user `b9a7593adca849d49a95cec18d31ee14`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:13.414504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7b2fe70a-e8be-4420-ada5-c78455ff90b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:13.486361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:13.502523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1180/2378] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:13.512089 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:13.515294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:13.580504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:13.600123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6da0ae-3226-4612-94b3-58e8ab07bab6 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:13.604185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddddedb-4990-4e3b-9ced-a9df19538826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:13.604842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddddedb-4990-4e3b-9ced-a9df19538826 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:13.605118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddddedb-4990-4e3b-9ced-a9df19538826 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:13.605513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddddedb-4990-4e3b-9ced-a9df19538826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:13.618504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddddedb-4990-4e3b-9ced-a9df19538826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:13.620243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddddedb-4990-4e3b-9ced-a9df19538826 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:13.632196 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1199/2379] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:14.655697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa046c8b-a0b3-464d-9c28-f774f9445f87 None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:14.659049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa046c8b-a0b3-464d-9c28-f774f9445f87 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:14.659563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa046c8b-a0b3-464d-9c28-f774f9445f87 None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:14.661555 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1181/2380] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:53:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:53:14.669417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:14.670413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:14.670992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:14.726563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa046c8b-a0b3-464d-9c28-f774f9445f87 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:14.728549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fa046c8b-a0b3-464d-9c28-f774f9445f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:14.753711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa046c8b-a0b3-464d-9c28-f774f9445f87 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:14.759481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:14.760645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:14.761107 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:14.761850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] PATH_INFO: `/v3/users/17123d3e5fe945bbb1cb835f154965d4` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:14.765508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] RBAC: Authorizing `identity:delete_user(user_id=17123d3e5fe945bbb1cb835f154965d4)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:14.766740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] MFA Rules not processed for user `2e9bc6cff2d9453a80cfce2c7e66d2e7`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:14.767666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:14.768191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:14.859894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1200/2381] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:53:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:14.877754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:14.879753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:14.890872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17123d3e5fe945bbb1cb835f154965d4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:14.891588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17123d3e5fe945bbb1cb835f154965d4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:14.916786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e60863-0127-41cd-b75c-26138bf60675 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:14.922820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681fb08-fe71-4591-ae36-ba853d10c169 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:14.923825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681fb08-fe71-4591-ae36-ba853d10c169 service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:14.924380 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681fb08-fe71-4591-ae36-ba853d10c169 service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:14.925123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681fb08-fe71-4591-ae36-ba853d10c169 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:14.927941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7681fb08-fe71-4591-ae36-ba853d10c169 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:14.929360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17123d3e5fe945bbb1cb835f154965d4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:14.940204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17123d3e5fe945bbb1cb835f154965d4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:14.940943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17123d3e5fe945bbb1cb835f154965d4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:14.945494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681fb08-fe71-4591-ae36-ba853d10c169 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:14.947223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681fb08-fe71-4591-ae36-ba853d10c169 service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:14.965631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1201/2382] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:14.965853 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1182/2383] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:53:14 2022] DELETE /identity/v3/users/17123d3e5fe945bbb1cb835f154965d4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:53:15.124878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb53a1-37c4-4003-bed9-e6b09b52ebc6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:15.125171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb53a1-37c4-4003-bed9-e6b09b52ebc6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:15.125472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb53a1-37c4-4003-bed9-e6b09b52ebc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:15.152501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-22cb53a1-37c4-4003-bed9-e6b09b52ebc6 None None] MFA Rules not processed for user `b9a7593adca849d49a95cec18d31ee14`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:15.153872 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-22cb53a1-37c4-4003-bed9-e6b09b52ebc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:15.215174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1202/2384] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:53:15.980362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:15.981911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:16.065022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e07a0-91cb-4162-9af5-ac1110f3012d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.070024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.070909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.071188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.071602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] PATH_INFO: `/v3/projects/cd4585e1e7524dd78f3cf187684ee2ba` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.072846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd4585e1e7524dd78f3cf187684ee2ba)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.074961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.165755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd4585e1e7524dd78f3cf187684ee2ba', 'request_id': 'req-f1bd9450-e14c-4bd6-be16-22397b9b31a6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:16.180225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd4585e1e7524dd78f3cf187684ee2ba', 'request_id': 'req-f1bd9450-e14c-4bd6-be16-22397b9b31a6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:16.187795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd4585e1e7524dd78f3cf187684ee2ba', 'request_id': 'req-f1bd9450-e14c-4bd6-be16-22397b9b31a6'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:16.201194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:16.215446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] The token cache is being invalidate because project cd4585e1e7524dd78f3cf187684ee2ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:53:16.215446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:16.218508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f1bd9450-e14c-4bd6-be16-22397b9b31a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:53:16.220311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1183/2385] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:53:15 2022] DELETE /identity/v3/projects/cd4585e1e7524dd78f3cf187684ee2ba => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:53:16.226668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb04bed-1abf-4348-8313-33c61f2a6e2f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.227183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb04bed-1abf-4348-8313-33c61f2a6e2f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.227411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb04bed-1abf-4348-8313-33c61f2a6e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.249275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7fb04bed-1abf-4348-8313-33c61f2a6e2f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:16.250784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7fb04bed-1abf-4348-8313-33c61f2a6e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:16.267053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:16.323448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1203/2386] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:16.331558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21c45-3c77-4013-befc-6093c0260e4a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.331772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21c45-3c77-4013-befc-6093c0260e4a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.332092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b21c45-3c77-4013-befc-6093c0260e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.351527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-38b21c45-3c77-4013-befc-6093c0260e4a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:16.352903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-38b21c45-3c77-4013-befc-6093c0260e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:16.361313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1184/2387] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:53:16.365499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb04bed-1abf-4348-8313-33c61f2a6e2f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:16.376469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb04bed-1abf-4348-8313-33c61f2a6e2f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.389361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.389899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.390185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.390461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.391632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:53:16.391632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.393391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.408035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1204/2388] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:53:16.410141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b21c45-3c77-4013-befc-6093c0260e4a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:16.421282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b21c45-3c77-4013-befc-6093c0260e4a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.426357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.427102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.427453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.427798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.428992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.431105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.460076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1185/2389] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:53:16 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:53:16.464245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:16.475183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043e6e9-d2d9-453c-86ac-c0adca16b5d4 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.541870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.542647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.543014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.543353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.545062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.547636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.600396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1205/2390] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:53:16 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:53:16.604492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:16.616731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3df05-4088-4042-8683-c56b7bae9e35 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.621768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.622451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.622770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.623177 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.623912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.625570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.639915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1186/2391] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:53:16.641352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:16.652941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ce49-720f-4279-93fd-3ee8bd8716f6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.657963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b46d41-0ec4-4703-9f42-e656a3212c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.658625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46d41-0ec4-4703-9f42-e656a3212c87 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.658906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46d41-0ec4-4703-9f42-e656a3212c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.659176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b46d41-0ec4-4703-9f42-e656a3212c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.659901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b46d41-0ec4-4703-9f42-e656a3212c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.661537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b46d41-0ec4-4703-9f42-e656a3212c87 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.672178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1206/2392] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:53:16.676433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:16.687168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511fb20c-945e-4a33-ad16-744200c841e3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.691783 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.692719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.693005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.693282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] PATH_INFO: `/v3/projects/e1f6fecd9b364f3bbc6281bf3a2e8ade/users/b8203a321cce4c558a69ff6ca75a4d27/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.708618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f6fecd9b364f3bbc6281bf3a2e8ade, user_id=b8203a321cce4c558a69ff6ca75a4d27, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.711262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.727687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1187/2393] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:53:16 2022] PUT /identity/v3/projects/e1f6fecd9b364f3bbc6281bf3a2e8ade/users/b8203a321cce4c558a69ff6ca75a4d27/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:53:16.732877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6ed53-bbc9-4c97-ad99-b0b17db2dbca None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.733259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6ed53-bbc9-4c97-ad99-b0b17db2dbca None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.733593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6ed53-bbc9-4c97-ad99-b0b17db2dbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.753061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-14a6ed53-bbc9-4c97-ad99-b0b17db2dbca None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:16.754457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-14a6ed53-bbc9-4c97-ad99-b0b17db2dbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:16.820934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1207/2394] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:53:16.828787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:16.830619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:16.892097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64055350-567a-4959-ac24-8d98a8da6e3a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:16.895732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:16.896414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:16.896698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:16.897004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:16.909083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:16.910659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:16.921855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1188/2395] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:19.689598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d2aae-d073-4bd5-ac14-94c83780d9b6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:19.690160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d2aae-d073-4bd5-ac14-94c83780d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:19.690256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d2aae-d073-4bd5-ac14-94c83780d9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:19.711269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fe2d2aae-d073-4bd5-ac14-94c83780d9b6 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:19.712908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fe2d2aae-d073-4bd5-ac14-94c83780d9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:19.748071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:19.802895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1208/2396] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:53:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:19.823962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:19.826407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:19.945876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965c5c-b78a-43ce-801b-5eec6a83329e service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:19.952139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b54a7-1899-4f2a-abbc-42541e643c05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:19.952859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b54a7-1899-4f2a-abbc-42541e643c05 service swift] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:19.953324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b54a7-1899-4f2a-abbc-42541e643c05 service swift] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:19.954518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b54a7-1899-4f2a-abbc-42541e643c05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:19.967236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b54a7-1899-4f2a-abbc-42541e643c05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:19.970403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b54a7-1899-4f2a-abbc-42541e643c05 service swift] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:19.986109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1189/2397] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:53:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:53:21.713435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227de05-5980-48bc-a3b9-10233c947a85 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:21.714051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227de05-5980-48bc-a3b9-10233c947a85 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:21.714051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227de05-5980-48bc-a3b9-10233c947a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:21.754135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9227de05-5980-48bc-a3b9-10233c947a85 None None] MFA Rules not processed for user `2de768b55f9a495295113de2ce1279e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:21.756775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9227de05-5980-48bc-a3b9-10233c947a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:21.825536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1209/2398] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:22.627769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26ada9-74ac-4cbc-ae58-548505146c93 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:22.628280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26ada9-74ac-4cbc-ae58-548505146c93 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:22.628377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26ada9-74ac-4cbc-ae58-548505146c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:22.691645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-fe26ada9-74ac-4cbc-ae58-548505146c93 None None] MFA Rules not processed for user `b8203a321cce4c558a69ff6ca75a4d27`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:22.693273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fe26ada9-74ac-4cbc-ae58-548505146c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:22.718754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:22.808744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1190/2399] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:22.813812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227de05-5980-48bc-a3b9-10233c947a85 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:22.829753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227de05-5980-48bc-a3b9-10233c947a85 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:22.842660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:22.843909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:22.844601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:22.846019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:22.847366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:22.850754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:22.916749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1210/2400] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:53:22 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:53:22.920913 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26ada9-74ac-4cbc-ae58-548505146c93 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:22.943035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26ada9-74ac-4cbc-ae58-548505146c93 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:23.031646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:23.032375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:23.032659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:23.032965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:23.033691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:23.035835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:23.072054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1191/2401] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:53:22 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:53:23.076152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:23.096653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8b5e-8b9e-469e-960c-b62a06a8ed02 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:23.105489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:23.106526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:23.107062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:23.107696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:23.109600 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:23.113054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:23.134218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1211/2402] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:53:23.140223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:23.151188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4c60f-9005-4241-be99-32f8bf113ff8 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:23.163723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:23.164770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:23.165095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:23.165375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] PATH_INFO: `/v3/projects/d74bc29e60ff431fb5536dcddccd39cd/users/fe3e3046e053405fa43f401688c5aa02/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:23.182219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74bc29e60ff431fb5536dcddccd39cd, user_id=fe3e3046e053405fa43f401688c5aa02, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:23.198896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:23.222166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1192/2403] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:53:23 2022] PUT /identity/v3/projects/d74bc29e60ff431fb5536dcddccd39cd/users/fe3e3046e053405fa43f401688c5aa02/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:53:23.223638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:23.252682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb7d7b-1c38-4ac1-b3b2-d787eeb54ae8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:23.369727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf328d43-4c15-4423-88e2-76ddd2fae350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:23.370418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328d43-4c15-4423-88e2-76ddd2fae350 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:23.370705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328d43-4c15-4423-88e2-76ddd2fae350 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:23.371037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf328d43-4c15-4423-88e2-76ddd2fae350 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:23.371775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf328d43-4c15-4423-88e2-76ddd2fae350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:23.373459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf328d43-4c15-4423-88e2-76ddd2fae350 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:23.391113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1212/2404] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:53:23.395340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:23.415693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f576004-d999-4b74-bb48-d50e8f46caa3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:23.421417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:23.422335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:23.422619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:23.422895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] PATH_INFO: `/v3/projects/d74bc29e60ff431fb5536dcddccd39cd/users/fe3e3046e053405fa43f401688c5aa02/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:23.427692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74bc29e60ff431fb5536dcddccd39cd, user_id=fe3e3046e053405fa43f401688c5aa02, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:23.432034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:23.454045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1193/2405] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:53:23 2022] PUT /identity/v3/projects/d74bc29e60ff431fb5536dcddccd39cd/users/fe3e3046e053405fa43f401688c5aa02/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:53:31.885899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43310c-60a0-4635-8266-2ad3a200b679 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:31.886724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43310c-60a0-4635-8266-2ad3a200b679 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:31.886724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43310c-60a0-4635-8266-2ad3a200b679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:31.931391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7b43310c-60a0-4635-8266-2ad3a200b679 None None] MFA Rules not processed for user `fe3e3046e053405fa43f401688c5aa02`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:31.936823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7b43310c-60a0-4635-8266-2ad3a200b679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:32.019423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:32.128082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1213/2406] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:32.139473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:32.142019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:32.276483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e820a2-e77e-449d-9983-809d40029ff9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:32.286368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:32.287266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:32.287612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:32.287895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:32.318629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:32.382706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:32.384393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:32.406932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1194/2407] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:32.595114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:32.595688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:32.596364 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:32.655384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:32.657357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:32.840688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1214/2408] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:53:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:32.841577 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:33.008773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa71ce3-222d-4808-80e7-5d95327106f9 service glance] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:33.046232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da39a6e-ed5b-4cab-9062-a52ee3bb9cc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:33.046972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da39a6e-ed5b-4cab-9062-a52ee3bb9cc4 service swift] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:33.047258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da39a6e-ed5b-4cab-9062-a52ee3bb9cc4 service swift] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:33.050060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da39a6e-ed5b-4cab-9062-a52ee3bb9cc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:33.081188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da39a6e-ed5b-4cab-9062-a52ee3bb9cc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:33.082773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da39a6e-ed5b-4cab-9062-a52ee3bb9cc4 service swift] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:33.108986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1195/2409] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:53:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:53:36.473110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:36.475695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:36.790663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda4d2c-5445-4f62-9c55-d52b3ce1851e None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:36.804835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1970e4-ce0c-413a-8c5c-5dd669eb8e8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:36.805635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1970e4-ce0c-413a-8c5c-5dd669eb8e8f service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:36.806720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1970e4-ce0c-413a-8c5c-5dd669eb8e8f service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:36.807895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1970e4-ce0c-413a-8c5c-5dd669eb8e8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:36.812120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-bb1970e4-ce0c-413a-8c5c-5dd669eb8e8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:36.855357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1970e4-ce0c-413a-8c5c-5dd669eb8e8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:36.859029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1970e4-ce0c-413a-8c5c-5dd669eb8e8f service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:36.899118 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1215/2410] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:43.096265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=49 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:45.457179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc696-9fd8-4d3c-807d-c69d87062cb0 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:45.457749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc696-9fd8-4d3c-807d-c69d87062cb0 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:45.457843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc696-9fd8-4d3c-807d-c69d87062cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:45.476246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c39dc696-9fd8-4d3c-807d-c69d87062cb0 None None] MFA Rules not processed for user `b8203a321cce4c558a69ff6ca75a4d27`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:45.477476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c39dc696-9fd8-4d3c-807d-c69d87062cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:45.485191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1196/2411] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:46.032376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f400c7-af1f-4e12-b888-6ae287582b5d None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.032946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f400c7-af1f-4e12-b888-6ae287582b5d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.032946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f400c7-af1f-4e12-b888-6ae287582b5d None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.034394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1216/2412] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:53:46.040664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e906b-ddc6-4aaf-9188-77b1b97afbdc None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.040952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e906b-ddc6-4aaf-9188-77b1b97afbdc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.041227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e906b-ddc6-4aaf-9188-77b1b97afbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.102701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-740e906b-ddc6-4aaf-9188-77b1b97afbdc None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:46.104310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-740e906b-ddc6-4aaf-9188-77b1b97afbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:46.226046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1197/2413] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:46.245229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f400c7-af1f-4e12-b888-6ae287582b5d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:46.273837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f400c7-af1f-4e12-b888-6ae287582b5d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:46.279221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaee5d8-f410-4531-b729-dc0711b862a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:46.280014 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaee5d8-f410-4531-b729-dc0711b862a9 service glance] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.280403 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaee5d8-f410-4531-b729-dc0711b862a9 service glance] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.280771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaee5d8-f410-4531-b729-dc0711b862a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.295012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaee5d8-f410-4531-b729-dc0711b862a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:46.296783 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaee5d8-f410-4531-b729-dc0711b862a9 service glance] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:46.309963 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1217/2414] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:46.400966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e7e56-498e-4ebc-a3e6-a8ea43a8687c None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.401395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e7e56-498e-4ebc-a3e6-a8ea43a8687c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.402153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e7e56-498e-4ebc-a3e6-a8ea43a8687c None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.404339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1198/2415] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:53:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 15:53:46.409980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51bdf-b76c-40cf-9dbd-3df3d88cdee5 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.410398 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51bdf-b76c-40cf-9dbd-3df3d88cdee5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.410980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a51bdf-b76c-40cf-9dbd-3df3d88cdee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.432232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-b7a51bdf-b76c-40cf-9dbd-3df3d88cdee5 None None] MFA Rules not processed for user `5e17d25a83fe48d99ddbd209fbea8486`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:46.433801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b7a51bdf-b76c-40cf-9dbd-3df3d88cdee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:46.476385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:46.506611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1218/2416] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:46.522023 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e7e56-498e-4ebc-a3e6-a8ea43a8687c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:46.525098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6d9e7e56-498e-4ebc-a3e6-a8ea43a8687c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:46.542788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e7e56-498e-4ebc-a3e6-a8ea43a8687c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:46.547395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa3130-1afa-41c3-b041-8306abd1daa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:46.548310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa3130-1afa-41c3-b041-8306abd1daa5 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.548595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa3130-1afa-41c3-b041-8306abd1daa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.548875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa3130-1afa-41c3-b041-8306abd1daa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.558047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa3130-1afa-41c3-b041-8306abd1daa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:46.561505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa3130-1afa-41c3-b041-8306abd1daa5 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:46.578709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1199/2417] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:46.715743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e459-6f26-4be2-8bed-3afbf612156d None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.716124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e459-6f26-4be2-8bed-3afbf612156d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.716347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2e459-6f26-4be2-8bed-3afbf612156d None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.717744 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1219/2418] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:53:46.725327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304662-e1f4-4929-8264-c2aa1dbda169 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.725616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304662-e1f4-4929-8264-c2aa1dbda169 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.725902 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7304662-e1f4-4929-8264-c2aa1dbda169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.747224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c7304662-e1f4-4929-8264-c2aa1dbda169 None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:46.748799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c7304662-e1f4-4929-8264-c2aa1dbda169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:46.760805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1200/2419] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:53:46.773178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2e459-6f26-4be2-8bed-3afbf612156d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:46.791438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2e459-6f26-4be2-8bed-3afbf612156d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:46.796349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:46.796869 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:46.796992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:46.797395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:46.816462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:46.818013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:46.829712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1220/2420] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:49.156509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe51f3-642b-4b60-b0aa-db0963c2f683 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:49.156928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe51f3-642b-4b60-b0aa-db0963c2f683 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:49.157239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe51f3-642b-4b60-b0aa-db0963c2f683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:49.219848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d6fe51f3-642b-4b60-b0aa-db0963c2f683 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:53:49.221149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d6fe51f3-642b-4b60-b0aa-db0963c2f683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:49.239210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1201/2421] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:53:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:53:49.252565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:49.282460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e8df9-1422-49ef-bba9-2a7d5b601bd6 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:49.288786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7865b-9149-44c2-b279-6b6e29df5cb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:49.289663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7865b-9149-44c2-b279-6b6e29df5cb4 service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:49.290176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7865b-9149-44c2-b279-6b6e29df5cb4 service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:49.290583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7865b-9149-44c2-b279-6b6e29df5cb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:49.300634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7865b-9149-44c2-b279-6b6e29df5cb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:49.302468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7865b-9149-44c2-b279-6b6e29df5cb4 service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:49.312884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1221/2422] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:53:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:53:50.508335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe51f3-642b-4b60-b0aa-db0963c2f683 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:53:50.518620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe51f3-642b-4b60-b0aa-db0963c2f683 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:53:50.522608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:53:50.523279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:53:50.523567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:53:50.524022 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:53:50.525805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:53:50.576805 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:53:50.595532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:53:50.597579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:53:50.608165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1202/2423] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:53:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:53:59.309515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=25 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:54:00.604030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=6 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:54:03.818304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0f64b-c98d-42c8-b960-ed8d43616c6f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:54:03.818856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0f64b-c98d-42c8-b960-ed8d43616c6f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:54:03.818856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0f64b-c98d-42c8-b960-ed8d43616c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:54:03.837892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-11c0f64b-c98d-42c8-b960-ed8d43616c6f None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:54:03.839126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-11c0f64b-c98d-42c8-b960-ed8d43616c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:54:03.846662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1222/2424] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:54:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:54:03.858589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:54:03.870992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cfa0f7-6e1d-42cb-bdc3-c0cb683b4679 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:54:03.877081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:54:03.877736 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:54:03.878021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:54:03.878304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:54:03.887389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:54:03.889029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:54:03.897881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1203/2425] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:54:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:54:13.833165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:54:13.894203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=6 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:54:34.231383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db5336-14f1-4854-9197-ee6a3217ca95 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:54:34.231917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db5336-14f1-4854-9197-ee6a3217ca95 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:54:34.231917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db5336-14f1-4854-9197-ee6a3217ca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:54:34.256219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-36db5336-14f1-4854-9197-ee6a3217ca95 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:54:34.257439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-36db5336-14f1-4854-9197-ee6a3217ca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:54:34.264970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1223/2426] 10.222.0.50 () {58 vars in 967 bytes} [Thu Jun 30 15:54:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:54:34.286642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:54:34.298065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026ce9c-f918-4c85-ad32-94eb4ce48e24 service swift] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:54:34.304521 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65401018-4b62-4a72-9f1f-1d65638e997f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:54:34.305221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401018-4b62-4a72-9f1f-1d65638e997f service swift] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:54:34.305502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401018-4b62-4a72-9f1f-1d65638e997f service swift] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:54:34.305773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401018-4b62-4a72-9f1f-1d65638e997f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:54:34.314852 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65401018-4b62-4a72-9f1f-1d65638e997f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:54:34.316515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65401018-4b62-4a72-9f1f-1d65638e997f service swift] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:54:34.324897 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1204/2427] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:54:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:54:43.423423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db5336-14f1-4854-9197-ee6a3217ca95 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:54:43.433088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db5336-14f1-4854-9197-ee6a3217ca95 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:54:43.436849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9942f-e3d1-4051-a5a4-6f141992a5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:54:43.437478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9942f-e3d1-4051-a5a4-6f141992a5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:54:43.437822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9942f-e3d1-4051-a5a4-6f141992a5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:54:43.438228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9942f-e3d1-4051-a5a4-6f141992a5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:54:43.447898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9942f-e3d1-4051-a5a4-6f141992a5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:54:43.449362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9942f-e3d1-4051-a5a4-6f141992a5d4 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:54:43.459059 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1224/2428] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:54:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:54:44.321637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=6 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:54:53.455036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=9 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:54:59.640274 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c9428-06cd-42d1-9d3a-e4f701ee49d3 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:54:59.640911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c9428-06cd-42d1-9d3a-e4f701ee49d3 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:54:59.640911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c9428-06cd-42d1-9d3a-e4f701ee49d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:54:59.661816 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0f3c9428-06cd-42d1-9d3a-e4f701ee49d3 None None] MFA Rules not processed for user `b9a7593adca849d49a95cec18d31ee14`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:54:59.663176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0f3c9428-06cd-42d1-9d3a-e4f701ee49d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:54:59.725268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1205/2429] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:54:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:55:01.112002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71206d7-e497-4031-8aa2-52180ae011e5 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:01.112530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71206d7-e497-4031-8aa2-52180ae011e5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:01.112530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71206d7-e497-4031-8aa2-52180ae011e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:01.130766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-b71206d7-e497-4031-8aa2-52180ae011e5 None None] MFA Rules not processed for user `d30ffdb0374c4600a09476cfb3a07cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:55:01.131948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b71206d7-e497-4031-8aa2-52180ae011e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:01.187863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1225/2430] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:55:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:55:09.718478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=25 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:55:11.180644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=25 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:55:30.918600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c9428-06cd-42d1-9d3a-e4f701ee49d3 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:55:30.934257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c9428-06cd-42d1-9d3a-e4f701ee49d3 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:55:30.941354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c505903-22a1-40c9-aa93-d1a3e867fcfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:55:30.942269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c505903-22a1-40c9-aa93-d1a3e867fcfb service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:30.942663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c505903-22a1-40c9-aa93-d1a3e867fcfb service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:30.943093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c505903-22a1-40c9-aa93-d1a3e867fcfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:30.946372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5c505903-22a1-40c9-aa93-d1a3e867fcfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:30.962128 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c505903-22a1-40c9-aa93-d1a3e867fcfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:55:30.963560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c505903-22a1-40c9-aa93-d1a3e867fcfb service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:55:30.975120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1206/2431] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:55:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:55:31.023766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71206d7-e497-4031-8aa2-52180ae011e5 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:55:31.034227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71206d7-e497-4031-8aa2-52180ae011e5 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:55:31.038617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:55:31.039267 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:31.039541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:31.039809 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:31.041932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:31.057224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:55:31.058771 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:55:31.070320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1226/2432] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:55:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:55:36.903099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbfb93-f091-44e3-bec9-76c493a6b41a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:36.903642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbfb93-f091-44e3-bec9-76c493a6b41a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:36.903642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbfb93-f091-44e3-bec9-76c493a6b41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:36.925768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e0fbfb93-f091-44e3-bec9-76c493a6b41a None None] MFA Rules not processed for user `7f2f23e763704ce2b0969e24ffde9243`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:55:36.927705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e0fbfb93-f091-44e3-bec9-76c493a6b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:37.000838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1207/2433] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:55:41.065428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=6 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:55:41.699761 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:55:41.701442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:41.829012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3377-a098-4e29-9c33-23587f2be506 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:55:41.837635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761ef07-59a3-40ce-ab3e-1aace3e672fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:55:41.840379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ef07-59a3-40ce-ab3e-1aace3e672fb service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:41.840671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ef07-59a3-40ce-ab3e-1aace3e672fb service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:41.841015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761ef07-59a3-40ce-ab3e-1aace3e672fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:41.843676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2761ef07-59a3-40ce-ab3e-1aace3e672fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:41.962319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761ef07-59a3-40ce-ab3e-1aace3e672fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:55:41.962319 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761ef07-59a3-40ce-ab3e-1aace3e672fb service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:55:41.971523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1227/2434] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:55:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:55:46.992381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=31 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:55:49.161671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbfb93-f091-44e3-bec9-76c493a6b41a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:55:49.175935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbfb93-f091-44e3-bec9-76c493a6b41a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:55:49.180862 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:55:49.181529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:49.181846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:49.182083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:49.184511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:49.198081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:55:49.199930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:55:49.211327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1208/2435] 10.222.0.50 () {60 vars in 1396 bytes} [Thu Jun 30 15:55:49 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:55:51.466724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979da13-52c6-485d-9620-d05199a243c9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:51.467657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979da13-52c6-485d-9620-d05199a243c9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:51.468157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979da13-52c6-485d-9620-d05199a243c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:51.486714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-e979da13-52c6-485d-9620-d05199a243c9 None None] MFA Rules not processed for user `2f3c67a263b14158a01fd906a31696cd`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:55:51.487960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e979da13-52c6-485d-9620-d05199a243c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:51.494932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1228/2436] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:55:51.981249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:55:51.991049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f1c36-8ea6-4722-ace0-50a6c166d4eb service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:55:52.002104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:55:52.003046 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:52.003301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:52.003605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:52.005842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:52.017136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:55:52.018602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:55:52.030283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1209/2437] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:55:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:55:52.378737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979da13-52c6-485d-9620-d05199a243c9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:55:52.392766 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979da13-52c6-485d-9620-d05199a243c9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:55:52.399508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:55:52.400419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:52.400859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:52.401310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:52.404484 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:52.426148 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:55:52.427758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:55:52.438145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1229/2438] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:55:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:55:55.303409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:55:55.315700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a16c7-7175-4ab8-9bad-c9b34c839360 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:55:55.320602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:55:55.321541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:55:55.321983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:55:55.322625 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:55:55.325090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:55:55.338227 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:55:55.339834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:55:55.350761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1210/2439] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:55:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:01.424664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:01.435363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d9121-1be8-4f73-96b1-17506ab2b11f service placement] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:01.439401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:01.440060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:01.440339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:01.440990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:01.442695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:01.454956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:01.456749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:01.466040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1230/2440] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:04.721780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:04.723322 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:04.798949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b960f-8d85-4141-b45a-65efc9d427d2 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:04.803531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:04.804446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:04.804728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:04.805006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] PATH_INFO: `/v3/users/b9a7593adca849d49a95cec18d31ee14` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:04.806208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a7593adca849d49a95cec18d31ee14)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:04.808190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:04.860430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a7593adca849d49a95cec18d31ee14'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:04.861105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a7593adca849d49a95cec18d31ee14'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:04.874154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a7593adca849d49a95cec18d31ee14'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:04.882713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a7593adca849d49a95cec18d31ee14'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:04.883360 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-8072432d-52fc-4933-b294-b03e717d75e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a7593adca849d49a95cec18d31ee14'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:04.897624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1211/2441] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:04 2022] DELETE /identity/v3/users/b9a7593adca849d49a95cec18d31ee14 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:05.283105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:05.297069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b93d6a-58a4-4725-b50c-157ff9bf274d service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:05.385675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:05.386830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.387257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.387666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] PATH_INFO: `/v3/projects/cf17d9690bf54b98815f00334a3470bc` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.389052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf17d9690bf54b98815f00334a3470bc)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:05.391257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:05.394578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:05.451894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf17d9690bf54b98815f00334a3470bc', 'request_id': 'req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:05.459233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf17d9690bf54b98815f00334a3470bc', 'request_id': 'req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:05.473830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf17d9690bf54b98815f00334a3470bc', 'request_id': 'req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:05.490020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] The token cache is being invalidate because project cf17d9690bf54b98815f00334a3470bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:05.490361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:05.491405 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ae1aa46e-10cb-4a70-ae1a-2a7116ebba19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:05.496428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1231/2442] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:05 2022] DELETE /identity/v3/projects/cf17d9690bf54b98815f00334a3470bc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:05.499064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a28b-ab55-4029-ae9b-2a8f444860bd None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.499474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a28b-ab55-4029-ae9b-2a8f444860bd None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.499774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a28b-ab55-4029-ae9b-2a8f444860bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.518689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-2533a28b-ab55-4029-ae9b-2a8f444860bd None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:05.519892 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2533a28b-ab55-4029-ae9b-2a8f444860bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:05.564390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:05.599859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1212/2443] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:05.608159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6705fe-7c7d-489c-9d58-22f5f336607c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.608445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6705fe-7c7d-489c-9d58-22f5f336607c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.608762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6705fe-7c7d-489c-9d58-22f5f336607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.627529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9c6705fe-7c7d-489c-9d58-22f5f336607c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:05.628778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9c6705fe-7c7d-489c-9d58-22f5f336607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:05.637962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1232/2444] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:05.641830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533a28b-ab55-4029-ae9b-2a8f444860bd None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:05.651617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533a28b-ab55-4029-ae9b-2a8f444860bd None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:05.656511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:05.657220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.657504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.657778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.658510 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:56:05.658815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:05.660444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:05.673170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1213/2445] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:56:05.677899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6705fe-7c7d-489c-9d58-22f5f336607c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:05.687529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6705fe-7c7d-489c-9d58-22f5f336607c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:05.692287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:05.692940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.693221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.693496 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.694241 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:05.696343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:05.729179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1233/2446] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:05 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:05.730281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:05.740657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eb324-8227-4153-be95-0ab9ece59759 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:05.798829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:05.799556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.799860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.800215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.800934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:05.802960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:05.828818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1214/2447] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:05 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:56:05.833043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:05.843549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864402eb-4378-459a-8133-30ddff093195 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:05.849998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:05.850988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.851367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.851702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.852491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:05.854663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:05.866133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1234/2448] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:05.870679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:05.881485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc12e-1d84-48e3-83e9-988f5516cad7 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:05.886754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39b26a-dcb4-4ed1-ac1f-5e923866dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:05.887413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39b26a-dcb4-4ed1-ac1f-5e923866dc27 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.887696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39b26a-dcb4-4ed1-ac1f-5e923866dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.887968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39b26a-dcb4-4ed1-ac1f-5e923866dc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.888765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39b26a-dcb4-4ed1-ac1f-5e923866dc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:05.890446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39b26a-dcb4-4ed1-ac1f-5e923866dc27 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:05.901195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1215/2449] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:05.905472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:05.915727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d2c58-dbc8-4576-8183-462b7dcc057a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:05.921918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:05.922998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.923278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.923556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] PATH_INFO: `/v3/projects/cef556810b2744128846e870f3ee776a/users/2394e4657c9a402bb131d6807ef60499/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.940429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cef556810b2744128846e870f3ee776a, user_id=2394e4657c9a402bb131d6807ef60499, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:05.943116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:05.957488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1235/2450] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:05 2022] PUT /identity/v3/projects/cef556810b2744128846e870f3ee776a/users/2394e4657c9a402bb131d6807ef60499/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:05.967692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:05.968043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:05.968544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:05.999823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] MFA Rules not processed for user `2394e4657c9a402bb131d6807ef60499`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:06.001001 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:06.071395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1216/2451] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:06.088442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:06.090442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:06.155302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e69ee9-1f32-44fc-82b2-137112183b62 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:06.160629 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:06.161401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:06.161689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:06.161965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:06.171283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:06.173026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:06.183073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1236/2452] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:12.409723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:12.411503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:12.463905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:12.502818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eabb51-532b-42f0-83c5-08dfe0ada268 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:12.509978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f342a7-acad-478f-99cb-221aed8be7c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:12.510644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f342a7-acad-478f-99cb-221aed8be7c1 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:12.510925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f342a7-acad-478f-99cb-221aed8be7c1 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:12.511198 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f342a7-acad-478f-99cb-221aed8be7c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:12.513057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e3f342a7-acad-478f-99cb-221aed8be7c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:12.526911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f342a7-acad-478f-99cb-221aed8be7c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:12.528603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f342a7-acad-478f-99cb-221aed8be7c1 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:12.542668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1217/2453] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:14.404008 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:14.439788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:14.473257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a092-0a23-487e-8757-62a37f4be1bb service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:14.478684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:14.479616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:14.479867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:14.480172 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] PATH_INFO: `/v3/users/2394e4657c9a402bb131d6807ef60499` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:14.481520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2394e4657c9a402bb131d6807ef60499)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:14.483486 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:14.537676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2394e4657c9a402bb131d6807ef60499'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:14.546201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2394e4657c9a402bb131d6807ef60499'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:14.553388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2394e4657c9a402bb131d6807ef60499'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:14.554028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2394e4657c9a402bb131d6807ef60499'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:14.554636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2394e4657c9a402bb131d6807ef60499'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:14.578525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1237/2454] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:14 2022] DELETE /identity/v3/users/2394e4657c9a402bb131d6807ef60499 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:14.583446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:14.583939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:14.583939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:14.603476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:14.604702 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:14.665207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1218/2455] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:14.675194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:14.676819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:14.734305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d872f1-651a-41c9-aab2-0e795d8c4bfd admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:14.738157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d8599-1464-48a6-bdf1-7d9453fffe89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:14.738796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8599-1464-48a6-bdf1-7d9453fffe89 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:14.739112 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8599-1464-48a6-bdf1-7d9453fffe89 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:14.739374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d8599-1464-48a6-bdf1-7d9453fffe89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:14.748822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8599-1464-48a6-bdf1-7d9453fffe89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:14.750265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d8599-1464-48a6-bdf1-7d9453fffe89 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:14.759904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1238/2456] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:15.054961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.056470 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:15.069807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a83e0c-bf3c-4176-a106-0fbb492597fe None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.074948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.075860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.076178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.076488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] PATH_INFO: `/v3/projects/cef556810b2744128846e870f3ee776a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.077731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cef556810b2744128846e870f3ee776a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.079659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.136295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cef556810b2744128846e870f3ee776a', 'request_id': 'req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:15.145417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cef556810b2744128846e870f3ee776a', 'request_id': 'req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:15.151945 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cef556810b2744128846e870f3ee776a', 'request_id': 'req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:15.163826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] The token cache is being invalidate because project cef556810b2744128846e870f3ee776a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:15.164129 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:15.164748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-af7c51ec-7ff7-452d-89ba-b2fbef566ce4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:15.166306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1219/2457] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:15 2022] DELETE /identity/v3/projects/cef556810b2744128846e870f3ee776a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:15.171384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe949fb-ec3a-4902-8f8a-80506cea5b1a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.171668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe949fb-ec3a-4902-8f8a-80506cea5b1a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.171937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe949fb-ec3a-4902-8f8a-80506cea5b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.190994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-5fe949fb-ec3a-4902-8f8a-80506cea5b1a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:15.192328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5fe949fb-ec3a-4902-8f8a-80506cea5b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:15.238895 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:15.256011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1239/2458] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:15.265323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68112c-7545-415a-b82e-4c88854ba385 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.265613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68112c-7545-415a-b82e-4c88854ba385 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.265891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68112c-7545-415a-b82e-4c88854ba385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.285527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-bd68112c-7545-415a-b82e-4c88854ba385 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:15.286831 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bd68112c-7545-415a-b82e-4c88854ba385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:15.295665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1220/2459] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:15.299915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe949fb-ec3a-4902-8f8a-80506cea5b1a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.312646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe949fb-ec3a-4902-8f8a-80506cea5b1a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.328157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.328849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.329130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.329400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.330143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:56:15.330449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.332374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.346052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1240/2460] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:56:15.351737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68112c-7545-415a-b82e-4c88854ba385 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.367520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68112c-7545-415a-b82e-4c88854ba385 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.382611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.383297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.383605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.383908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.384666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.386661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.419873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1221/2461] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:15 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:15.425693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.436612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d7803-b5cd-415c-9ee3-03f7b9fbd2b0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.497637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.498548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.498944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.499341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.500385 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.503302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.530677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1241/2462] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:15 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:15.534936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.546032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ff88e-20a7-4da1-b7b3-825315a74f5b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.552093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.552764 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.553052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.553327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.554064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.555691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.566639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1222/2463] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:15.570626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.585362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593370bc-750d-4c45-af2b-5aeee94d41bb admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.590929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c1f16-133f-4cc1-b82b-7f619230ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.591603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c1f16-133f-4cc1-b82b-7f619230ee93 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.591879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c1f16-133f-4cc1-b82b-7f619230ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.592192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c1f16-133f-4cc1-b82b-7f619230ee93 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.592912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c1f16-133f-4cc1-b82b-7f619230ee93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.594518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c1f16-133f-4cc1-b82b-7f619230ee93 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.606892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1242/2464] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:15.611212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.619098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:15.623612 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf33133-5b81-4b86-8c9c-175b519b13ad admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.629583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.630881 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.631336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.631819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] PATH_INFO: `/v3/projects/7b94cd4341d64c409783fecfa81763e1/users/6e1730c8601c4334913b13e85e76e8d1/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.649583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b94cd4341d64c409783fecfa81763e1, user_id=6e1730c8601c4334913b13e85e76e8d1, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.653014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.676118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1223/2465] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:15 2022] PUT /identity/v3/projects/7b94cd4341d64c409783fecfa81763e1/users/6e1730c8601c4334913b13e85e76e8d1/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:15.679091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0091e-c167-4c16-9b11-7a4f10eab2ba None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.679386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0091e-c167-4c16-9b11-7a4f10eab2ba None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.679727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e0091e-c167-4c16-9b11-7a4f10eab2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.699858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-95e0091e-c167-4c16-9b11-7a4f10eab2ba None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:15.701214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-95e0091e-c167-4c16-9b11-7a4f10eab2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:15.769506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1243/2466] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:15.779704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:15.781609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:15.844257 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f33652-6378-4f36-9c21-1c1abbed13e9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:15.847887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd518-2abe-4f75-b42f-fb5565d89a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:15.848623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd518-2abe-4f75-b42f-fb5565d89a76 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:15.848901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd518-2abe-4f75-b42f-fb5565d89a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:15.849174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd518-2abe-4f75-b42f-fb5565d89a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:15.858610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fd518-2abe-4f75-b42f-fb5565d89a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:15.860098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fd518-2abe-4f75-b42f-fb5565d89a76 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:15.870205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1224/2467] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:19.164444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0091e-c167-4c16-9b11-7a4f10eab2ba None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:19.176392 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e0091e-c167-4c16-9b11-7a4f10eab2ba None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:19.189102 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:19.189921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:19.190834 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:19.191358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:19.194141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:19.210612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:19.212425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:19.225723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1244/2468] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:20.066889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e6be-73b2-41db-9035-6ff9c7e4cffc None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:20.067412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e6be-73b2-41db-9035-6ff9c7e4cffc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:20.067412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203e6be-73b2-41db-9035-6ff9c7e4cffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:20.096454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4203e6be-73b2-41db-9035-6ff9c7e4cffc None None] MFA Rules not processed for user `6e1730c8601c4334913b13e85e76e8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:20.098343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4203e6be-73b2-41db-9035-6ff9c7e4cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:20.175915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1225/2469] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:20.180565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:20.194897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77495d8-4d38-4731-a2d9-a85bea966490 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:20.208592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:20.209282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:20.209552 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:20.210068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:20.211117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:20.212954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:20.259341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1245/2470] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:20 2022] POST /identity/v3/projects => generated 371 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:56:20.263855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203e6be-73b2-41db-9035-6ff9c7e4cffc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:20.288077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203e6be-73b2-41db-9035-6ff9c7e4cffc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:20.383282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:20.384375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:20.384728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:20.385070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:20.385845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:20.387907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:20.431208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1226/2471] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:20 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:20.432725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:20.452088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30fb3-a2f0-4ff5-9691-2c52b936973f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:20.464668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:20.465271 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:20.465554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:20.465830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:20.466570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:20.468303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:20.485726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1246/2472] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:20.490246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:20.497456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:20.504942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37fc03-e147-48cc-9686-a59003392aa9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:20.510723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:20.511766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:20.512249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:20.512479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] PATH_INFO: `/v3/projects/5ca7585f7fca4966950cdce17727c505/users/0036d84503384235b2154dbb33ed9e52/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:20.530986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca7585f7fca4966950cdce17727c505, user_id=0036d84503384235b2154dbb33ed9e52, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:20.533868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:20.556343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1227/2473] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:20 2022] PUT /identity/v3/projects/5ca7585f7fca4966950cdce17727c505/users/0036d84503384235b2154dbb33ed9e52/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:20.560710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:20.570817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac0357-9dd8-4ad5-8f80-aafb4c296bd3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:20.582676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:20.649811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:20.650731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:20.650867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:20.651042 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:20.652011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:20.654079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:20.666303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1247/2474] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:20.670279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:20.681962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02727054-3cdc-40c2-8e60-0fb64386fe38 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:20.687377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:20.688036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:20.688393 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:20.688673 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] PATH_INFO: `/v3/projects/5ca7585f7fca4966950cdce17727c505/users/0036d84503384235b2154dbb33ed9e52/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:20.690674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca7585f7fca4966950cdce17727c505, user_id=0036d84503384235b2154dbb33ed9e52, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:20.694012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:20.706631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1228/2475] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:20 2022] PUT /identity/v3/projects/5ca7585f7fca4966950cdce17727c505/users/0036d84503384235b2154dbb33ed9e52/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:23.449856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:23.451762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:23.474787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc4cfe-3b21-4f52-ab38-1791b0f2b6e4 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:23.591397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:23.592828 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:23.594480 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:23.594998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] PATH_INFO: `/v3/users/d30ffdb0374c4600a09476cfb3a07cb4` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:23.598603 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d30ffdb0374c4600a09476cfb3a07cb4)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:23.601763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:23.707838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd30ffdb0374c4600a09476cfb3a07cb4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:23.723703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd30ffdb0374c4600a09476cfb3a07cb4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:23.761191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd30ffdb0374c4600a09476cfb3a07cb4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:23.770509 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd30ffdb0374c4600a09476cfb3a07cb4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:23.790040 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d47b32d7-b599-401b-97bc-798ddc2f3d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd30ffdb0374c4600a09476cfb3a07cb4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:23.832475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1248/2476] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:23 2022] DELETE /identity/v3/users/d30ffdb0374c4600a09476cfb3a07cb4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:24.174327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:24.177847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:24.200237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47028fb-3e68-4e39-bb60-3e8a97aa93f9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:24.309998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:24.310961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:24.311454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:24.312618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] PATH_INFO: `/v3/projects/01a8c235ed97450bb8518af9d7dc1983` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:24.314919 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a8c235ed97450bb8518af9d7dc1983)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:24.317566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:24.416418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a8c235ed97450bb8518af9d7dc1983', 'request_id': 'req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:24.443333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a8c235ed97450bb8518af9d7dc1983', 'request_id': 'req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:24.460680 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a8c235ed97450bb8518af9d7dc1983', 'request_id': 'req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:24.499790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] The token cache is being invalidate because project 01a8c235ed97450bb8518af9d7dc1983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:24.500123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:24.502595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-7f9dad57-04c4-4f1b-8a88-f94f8ebbf10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:24.506323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1229/2477] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:24 2022] DELETE /identity/v3/projects/01a8c235ed97450bb8518af9d7dc1983 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:24.511512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199adc10-8e7d-4d64-9194-ce6523b09b72 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:24.512046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199adc10-8e7d-4d64-9194-ce6523b09b72 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:24.512136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199adc10-8e7d-4d64-9194-ce6523b09b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:24.541135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-199adc10-8e7d-4d64-9194-ce6523b09b72 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:24.544114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-199adc10-8e7d-4d64-9194-ce6523b09b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:24.575321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:24.632811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1249/2478] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:24.640453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc92133-451d-4cc2-8bff-d401188a4811 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:24.640748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc92133-451d-4cc2-8bff-d401188a4811 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:24.641021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc92133-451d-4cc2-8bff-d401188a4811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:24.682854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4cc92133-451d-4cc2-8bff-d401188a4811 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:24.684950 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4cc92133-451d-4cc2-8bff-d401188a4811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:24.709745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1230/2479] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:24.714979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199adc10-8e7d-4d64-9194-ce6523b09b72 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:24.729438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199adc10-8e7d-4d64-9194-ce6523b09b72 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:24.741973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:24.742672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:24.742956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:24.743235 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:24.744682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:56:24.744997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:24.746673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:24.775291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1250/2480] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:56:24.779586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc92133-451d-4cc2-8bff-d401188a4811 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:24.791951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc92133-451d-4cc2-8bff-d401188a4811 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:24.816859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:24.818043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:24.818697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:24.819130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:24.820066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:24.822205 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:24.894342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1231/2481] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:24 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:24.896417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:24.914106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6558e-e528-45e8-8c03-7097bb149af4 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.050485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.051202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.051483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.051755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.052513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.054548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.103494 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1251/2482] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:24 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:25.107182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:25.123407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca25f8f-d583-423d-98a4-94cab0e516b6 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.130851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.131866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.133196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.133838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.135972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.140311 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.171553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1232/2483] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:25.173147 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:25.185867 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57082ae7-5033-49b2-9340-02af85659508 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.191162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab66c46-9801-49c1-a593-93629bf7faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.192067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab66c46-9801-49c1-a593-93629bf7faf8 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.192452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab66c46-9801-49c1-a593-93629bf7faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.192835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab66c46-9801-49c1-a593-93629bf7faf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.193690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab66c46-9801-49c1-a593-93629bf7faf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.195961 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab66c46-9801-49c1-a593-93629bf7faf8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.217410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1252/2484] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:25.219017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:25.235790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614ad28-19d5-4169-b205-d7c6be22c5de admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.244510 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.246237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.246732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.247958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] PATH_INFO: `/v3/projects/e7d11d59575e4bca89293162bb0c3bb0/users/476cda17bb184296817b8216d85cb81a/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.275212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:25.293203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d11d59575e4bca89293162bb0c3bb0, user_id=476cda17bb184296817b8216d85cb81a, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.304388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.338352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1233/2485] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:25 2022] PUT /identity/v3/projects/e7d11d59575e4bca89293162bb0c3bb0/users/476cda17bb184296817b8216d85cb81a/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:25.340114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f88ab-0653-4aa7-bf38-f2c79b9b01d5 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.340410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f88ab-0653-4aa7-bf38-f2c79b9b01d5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.340694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f88ab-0653-4aa7-bf38-f2c79b9b01d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.374580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-950f88ab-0653-4aa7-bf38-f2c79b9b01d5 None None] MFA Rules not processed for user `476cda17bb184296817b8216d85cb81a`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:25.376576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-950f88ab-0653-4aa7-bf38-f2c79b9b01d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:25.474579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1253/2486] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:25.479299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:25.569579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05028224-97df-47e3-8d6d-ef3c459a9cdd admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.576966 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.577655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.577933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.579366 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.580138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.582250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.624955 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1234/2487] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:25 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:25.630325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f88ab-0653-4aa7-bf38-f2c79b9b01d5 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:25.647117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f88ab-0653-4aa7-bf38-f2c79b9b01d5 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.681786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:25.746162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.747272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.749585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.750845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.751699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.759081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.808798 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1254/2488] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:25 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:25.811105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:25.825784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99855160-fe3b-45b6-bdee-00a35c34df43 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.831957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.832908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.834527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.834527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.834527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.836534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.865194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1235/2489] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:25.869810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:25.882588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98eab4b-3c1d-4c7e-bc25-3632561f6822 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:25.899479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:25.901510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:25.901510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:25.901821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] PATH_INFO: `/v3/projects/8f6d4a56c1b642a7a61d855045f63bc3/users/1f566b120a1041128ed0d550841016f4/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:25.948856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6d4a56c1b642a7a61d855045f63bc3, user_id=1f566b120a1041128ed0d550841016f4, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:25.951726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:25.987773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1255/2490] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:25 2022] PUT /identity/v3/projects/8f6d4a56c1b642a7a61d855045f63bc3/users/1f566b120a1041128ed0d550841016f4/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:25.991554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:26.007133 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de198483-da09-48c7-aed3-fc3c0bf53523 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:26.131045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f491416-4b22-4b23-88ca-5f51f7709da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:26.132036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f491416-4b22-4b23-88ca-5f51f7709da2 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:26.132316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f491416-4b22-4b23-88ca-5f51f7709da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:26.132651 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f491416-4b22-4b23-88ca-5f51f7709da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:26.133460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f491416-4b22-4b23-88ca-5f51f7709da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:26.137065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f491416-4b22-4b23-88ca-5f51f7709da2 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:26.165119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1236/2491] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:26.169988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:26.184709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9c11e-f2a2-4844-ac40-cd2c4e1b6db0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:26.190103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:26.191092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:26.191437 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:26.191809 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] PATH_INFO: `/v3/projects/8f6d4a56c1b642a7a61d855045f63bc3/users/1f566b120a1041128ed0d550841016f4/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:26.194700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6d4a56c1b642a7a61d855045f63bc3, user_id=1f566b120a1041128ed0d550841016f4, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:26.197743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:26.210731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1256/2492] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:26 2022] PUT /identity/v3/projects/8f6d4a56c1b642a7a61d855045f63bc3/users/1f566b120a1041128ed0d550841016f4/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:26.220882 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b53de-722a-4039-82eb-36cad37e6752 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:26.221358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b53de-722a-4039-82eb-36cad37e6752 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:26.221466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b53de-722a-4039-82eb-36cad37e6752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:26.290281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a09b53de-722a-4039-82eb-36cad37e6752 None None] MFA Rules not processed for user `1f566b120a1041128ed0d550841016f4`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:26.291538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a09b53de-722a-4039-82eb-36cad37e6752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:26.422468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:26.450211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1237/2493] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:26 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:26.464529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:26.468610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:26.610680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90e370-2520-4cf8-bb90-e8bc98d97844 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:26.614571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:26.615472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:26.615825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:26.616197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:26.627352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:26.629027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:26.649053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1257/2494] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:26 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:27.072403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3110-fe31-4ab0-9a04-9547a20aae5f None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:27.072786 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3110-fe31-4ab0-9a04-9547a20aae5f None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:27.073172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b3110-fe31-4ab0-9a04-9547a20aae5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:27.120995 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6f5b3110-fe31-4ab0-9a04-9547a20aae5f None None] MFA Rules not processed for user `0036d84503384235b2154dbb33ed9e52`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:27.122348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6f5b3110-fe31-4ab0-9a04-9547a20aae5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:27.248055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1238/2495] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:27.276944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:27.278926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:27.328289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b3110-fe31-4ab0-9a04-9547a20aae5f None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:27.389342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227154b6-5543-4dcc-b1bb-f4b3ba2659fa service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:27.406791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c2b94-531c-48dd-b089-783b4067cf21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:27.407471 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2b94-531c-48dd-b089-783b4067cf21 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:27.407753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2b94-531c-48dd-b089-783b4067cf21 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:27.408049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c2b94-531c-48dd-b089-783b4067cf21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:27.409936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-832c2b94-531c-48dd-b089-783b4067cf21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:27.430731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b3110-fe31-4ab0-9a04-9547a20aae5f None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:27.431434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:27.436076 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:27.437327 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:27.437686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:27.437944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] PATH_INFO: `/v3/users/476cda17bb184296817b8216d85cb81a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:27.439670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=476cda17bb184296817b8216d85cb81a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:27.441735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:27.495954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c2b94-531c-48dd-b089-783b4067cf21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:27.497884 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c2b94-531c-48dd-b089-783b4067cf21 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:27.508498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1258/2496] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:27.510286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:27.510701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:27.511052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:27.511717 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '476cda17bb184296817b8216d85cb81a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:27.512496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '476cda17bb184296817b8216d85cb81a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:27.527586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '476cda17bb184296817b8216d85cb81a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:27.532586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] MFA Rules not processed for user `2de768b55f9a495295113de2ce1279e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:27.534044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:27.542071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '476cda17bb184296817b8216d85cb81a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:27.543105 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1f67e76a-8fbf-4e4c-97c1-2103f8490128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '476cda17bb184296817b8216d85cb81a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:27.545798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:27.564086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1239/2497] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:27 2022] DELETE /identity/v3/users/476cda17bb184296817b8216d85cb81a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:27.567915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:27.574524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:27.574819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:27.607306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:27.608788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:27.614836 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1259/2498] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:27.768441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1240/2499] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:27.791021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:27.792874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:27.921919 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aafd1cb-16b4-4d46-91cb-4b77ceac052a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:28.012222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:28.012935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:28.013217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:28.014173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:28.026090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:28.028132 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:28.047718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1260/2500] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:56:28.592232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:28.593951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:28.610350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086cd59-e65c-47af-95bb-2d51403d79bc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:28.623604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:28.624989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:28.625279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:28.625557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] PATH_INFO: `/v3/projects/e7d11d59575e4bca89293162bb0c3bb0` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:28.632823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7d11d59575e4bca89293162bb0c3bb0)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:28.636209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:28.698031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7d11d59575e4bca89293162bb0c3bb0', 'request_id': 'req-78c7a275-b428-471c-87e3-040dd4788f57'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.719464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7d11d59575e4bca89293162bb0c3bb0', 'request_id': 'req-78c7a275-b428-471c-87e3-040dd4788f57'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.726973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7d11d59575e4bca89293162bb0c3bb0', 'request_id': 'req-78c7a275-b428-471c-87e3-040dd4788f57'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.750332 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] The token cache is being invalidate because project e7d11d59575e4bca89293162bb0c3bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:28.750735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.751501 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-78c7a275-b428-471c-87e3-040dd4788f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.757494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1241/2501] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:28 2022] DELETE /identity/v3/projects/e7d11d59575e4bca89293162bb0c3bb0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:28.757843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:28.770997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025fb42-7c3c-4444-b38a-54c582488c27 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:28.805582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:28.867679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:28.870090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:28.871330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:28.871831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] PATH_INFO: `/v3/users/1f566b120a1041128ed0d550841016f4` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:28.874841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f566b120a1041128ed0d550841016f4)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:28.878299 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:28.955590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f566b120a1041128ed0d550841016f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.970984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f566b120a1041128ed0d550841016f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.984536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f566b120a1041128ed0d550841016f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.986135 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f566b120a1041128ed0d550841016f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:28.986894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f566b120a1041128ed0d550841016f4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:29.020083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1261/2502] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:28 2022] DELETE /identity/v3/users/1f566b120a1041128ed0d550841016f4 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:29.256756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e4776-95b5-4624-a6ec-c225f3920d28 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.257074 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e4776-95b5-4624-a6ec-c225f3920d28 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.257381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e4776-95b5-4624-a6ec-c225f3920d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.288367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-9f6e4776-95b5-4624-a6ec-c225f3920d28 None None] MFA Rules not processed for user `6e1730c8601c4334913b13e85e76e8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:29.289638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9f6e4776-95b5-4624-a6ec-c225f3920d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:29.358345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:29.360194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:29.379804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1242/2503] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:29.446253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a3e83-149e-4c47-864e-7cbbadf13e6a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:29.452418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:29.453400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.453776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.454277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] PATH_INFO: `/v3/projects/8f6d4a56c1b642a7a61d855045f63bc3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.455428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6d4a56c1b642a7a61d855045f63bc3)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:29.459179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:29.511662 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:29.521254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6d4a56c1b642a7a61d855045f63bc3', 'request_id': 'req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:29.528451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6d4a56c1b642a7a61d855045f63bc3', 'request_id': 'req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:29.539681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6d4a56c1b642a7a61d855045f63bc3', 'request_id': 'req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:29.551336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] The token cache is being invalidate because project 8f6d4a56c1b642a7a61d855045f63bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:29.551694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:29.552413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-36e4c63b-fda0-41ec-aa4c-7a325c44f62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:29.554193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1262/2504] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:29 2022] DELETE /identity/v3/projects/8f6d4a56c1b642a7a61d855045f63bc3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:29.559306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38193-530b-45ea-a7b5-c229a2645507 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.559600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38193-530b-45ea-a7b5-c229a2645507 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.559877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38193-530b-45ea-a7b5-c229a2645507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.581253 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5bf38193-530b-45ea-a7b5-c229a2645507 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:29.583548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5bf38193-530b-45ea-a7b5-c229a2645507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:29.598922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:29.649824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1243/2505] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:29.659519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c154a7-0a79-40e4-a58d-a1b9ffc0073b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.659806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c154a7-0a79-40e4-a58d-a1b9ffc0073b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.660205 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c154a7-0a79-40e4-a58d-a1b9ffc0073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.680739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-13c154a7-0a79-40e4-a58d-a1b9ffc0073b None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:29.682087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-13c154a7-0a79-40e4-a58d-a1b9ffc0073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:29.690588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1263/2506] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:29.695542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf38193-530b-45ea-a7b5-c229a2645507 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:29.706411 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf38193-530b-45ea-a7b5-c229a2645507 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:29.711014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:29.711690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.711969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.712269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.712999 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:56:29.713313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:29.714900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:29.727795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1244/2507] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:56:29.732284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c154a7-0a79-40e4-a58d-a1b9ffc0073b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:29.742960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c154a7-0a79-40e4-a58d-a1b9ffc0073b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:29.748732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:29.749457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.749740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.750067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.750813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:29.753048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:29.780742 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1264/2508] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:29 2022] POST /identity/v3/projects => generated 357 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:56:29.793286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:29.807000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24fe4d6-b115-4921-8f95-8f298f395b38 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:29.876313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:29.877307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.877827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.878299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.879260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:29.882134 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:29.907383 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1245/2509] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:29 2022] POST /identity/v3/users => generated 279 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:56:29.913815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:29.925512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf0488-eb42-467e-b3dc-2e30f8b407d7 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:29.931688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:29.932692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.933124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.933544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.934349 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:29.936015 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:29.949715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1265/2510] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:29.954021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:29.964505 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3093b01-ce2c-4e60-9533-e853d0006391 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:29.971560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956d4cf-74ad-4c67-a53d-87705a48cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:29.972784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956d4cf-74ad-4c67-a53d-87705a48cfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:29.973228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956d4cf-74ad-4c67-a53d-87705a48cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:29.973660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956d4cf-74ad-4c67-a53d-87705a48cfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:29.974795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956d4cf-74ad-4c67-a53d-87705a48cfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:29.976527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956d4cf-74ad-4c67-a53d-87705a48cfb8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:29.988542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1246/2511] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:29.992965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.003077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6113b58-b74f-4f6a-9c15-82741887d380 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:30.010387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c074e61-2161-40de-acde-936db262331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:30.011462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c074e61-2161-40de-acde-936db262331a admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.011812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c074e61-2161-40de-acde-936db262331a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.012322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c074e61-2161-40de-acde-936db262331a admin admin] PATH_INFO: `/v3/projects/9b5dab8d4a5e4feda1b3dd49375460ca/users/ffd06e916db54c61884c6f6aae64ddc0/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.035338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c074e61-2161-40de-acde-936db262331a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5dab8d4a5e4feda1b3dd49375460ca, user_id=ffd06e916db54c61884c6f6aae64ddc0, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:30.039413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c074e61-2161-40de-acde-936db262331a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:30.076993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1266/2512] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:29 2022] PUT /identity/v3/projects/9b5dab8d4a5e4feda1b3dd49375460ca/users/ffd06e916db54c61884c6f6aae64ddc0/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:30.087661 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f0c0-72cb-4b53-8c5b-a63c75669cc1 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.087939 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f0c0-72cb-4b53-8c5b-a63c75669cc1 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.088406 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895f0c0-72cb-4b53-8c5b-a63c75669cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.124511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8895f0c0-72cb-4b53-8c5b-a63c75669cc1 None None] MFA Rules not processed for user `ffd06e916db54c61884c6f6aae64ddc0`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:30.125721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8895f0c0-72cb-4b53-8c5b-a63c75669cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:30.204444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1247/2513] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:30.205652 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c074e61-2161-40de-acde-936db262331a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.286523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c074e61-2161-40de-acde-936db262331a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:30.291254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:30.291954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.292269 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.292542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.293250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:30.295193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:30.327251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1267/2514] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:30 2022] POST /identity/v3/projects => generated 357 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:30.331433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895f0c0-72cb-4b53-8c5b-a63c75669cc1 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.352037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895f0c0-72cb-4b53-8c5b-a63c75669cc1 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:30.372240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:30.453410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:30.454378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.454378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.454844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.455703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:30.457850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:30.488430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1248/2515] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:30 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:30.489773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.505544 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78903c-af40-423f-8a99-8c73297b0948 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:30.518659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:30.519607 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.520554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.520973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.522140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:30.525811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:30.544143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1268/2516] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:30.545792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.567678 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbda80-4f58-445e-95db-23bdd6128cb3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:30.572790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:30.574019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.574506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.574793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] PATH_INFO: `/v3/projects/736a2fd18780437eaaf1f512949dc73c/users/967b776969c84b47bb94b705eb688ea1/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.593638 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=736a2fd18780437eaaf1f512949dc73c, user_id=967b776969c84b47bb94b705eb688ea1, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:30.596509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:30.616034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1249/2517] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:30 2022] PUT /identity/v3/projects/736a2fd18780437eaaf1f512949dc73c/users/967b776969c84b47bb94b705eb688ea1/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:30.621575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.634250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666855a-8159-42e3-bd3b-2f1429fc2406 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:30.680208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:30.716928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3290f-a110-4d05-8f9f-42bec179b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:30.717659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3290f-a110-4d05-8f9f-42bec179b833 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.717937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3290f-a110-4d05-8f9f-42bec179b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.718206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3290f-a110-4d05-8f9f-42bec179b833 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.718993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3290f-a110-4d05-8f9f-42bec179b833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:30.721439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3290f-a110-4d05-8f9f-42bec179b833 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:30.733512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1269/2518] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:30.737770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.749242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391b8a1-737b-4bd2-b0a6-bf38c6add4e9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:30.754768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:30.755748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.756175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.756554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] PATH_INFO: `/v3/projects/736a2fd18780437eaaf1f512949dc73c/users/967b776969c84b47bb94b705eb688ea1/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.758829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=736a2fd18780437eaaf1f512949dc73c, user_id=967b776969c84b47bb94b705eb688ea1, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:30.761874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:30.773788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1250/2519] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:30 2022] PUT /identity/v3/projects/736a2fd18780437eaaf1f512949dc73c/users/967b776969c84b47bb94b705eb688ea1/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:30.784996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dd9f6-0879-4c2b-a21a-b4a74855d9e9 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:30.785583 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dd9f6-0879-4c2b-a21a-b4a74855d9e9 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:30.785917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330dd9f6-0879-4c2b-a21a-b4a74855d9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:30.814915 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-330dd9f6-0879-4c2b-a21a-b4a74855d9e9 None None] MFA Rules not processed for user `967b776969c84b47bb94b705eb688ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:30.816216 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-330dd9f6-0879-4c2b-a21a-b4a74855d9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:30.942106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1270/2520] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:30.957935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:30.962904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:31.052524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53412f2a-89b1-4961-8d81-8154b3f67d9e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:31.057268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:31.058158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:31.058440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:31.058721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:31.088231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:31.090176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:31.115371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1251/2521] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:30 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:31.230416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dd9f6-0879-4c2b-a21a-b4a74855d9e9 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:31.240748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330dd9f6-0879-4c2b-a21a-b4a74855d9e9 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:31.244781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:31.245510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:31.246034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:31.246501 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:31.311226 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:31.313033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:31.324712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1271/2522] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:31 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:37.089357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:37.091473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:37.189980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07411d99-c04a-4510-8ed9-3491025099a6 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:37.196432 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a003bbd-fa83-4e35-b36e-a3d50058ed1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:37.197207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003bbd-fa83-4e35-b36e-a3d50058ed1e service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:37.197634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003bbd-fa83-4e35-b36e-a3d50058ed1e service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:37.198015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a003bbd-fa83-4e35-b36e-a3d50058ed1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:37.201187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0a003bbd-fa83-4e35-b36e-a3d50058ed1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:37.213086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:37.217743 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a003bbd-fa83-4e35-b36e-a3d50058ed1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:37.219660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a003bbd-fa83-4e35-b36e-a3d50058ed1e service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:37.233153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1252/2523] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:41.319764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=73 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:41.911894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:41.913391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:41.929787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b235d42b-424a-4c8a-b2fa-c275e49f01ce service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:41.935590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:41.936279 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:41.936566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:41.936842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:41.938661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:41.952816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:41.954564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:41.968981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1272/2524] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:45.660084 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cab1-d53a-4e33-9b80-0b05f0443d73 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:45.660556 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cab1-d53a-4e33-9b80-0b05f0443d73 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:45.660670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cab1-d53a-4e33-9b80-0b05f0443d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:45.679182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-58c0cab1-d53a-4e33-9b80-0b05f0443d73 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:45.680653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-58c0cab1-d53a-4e33-9b80-0b05f0443d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:45.755170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1253/2525] 10.222.0.50 () {58 vars in 967 bytes} [Thu Jun 30 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:45.807835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:45.809455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:45.892613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67455619-4886-4341-9fb2-a3851122edf1 service nova] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:45.897427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bcc37-0136-4e02-a8ca-4abccd612fca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:45.898073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcc37-0136-4e02-a8ca-4abccd612fca service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:45.898354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcc37-0136-4e02-a8ca-4abccd612fca service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:45.898631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bcc37-0136-4e02-a8ca-4abccd612fca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:45.906993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bcc37-0136-4e02-a8ca-4abccd612fca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:45.908632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bcc37-0136-4e02-a8ca-4abccd612fca service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:45.916835 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1273/2526] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:56:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:56:46.726096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cab1-d53a-4e33-9b80-0b05f0443d73 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:46.813181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cab1-d53a-4e33-9b80-0b05f0443d73 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:46.818111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:46.819044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:46.819599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:46.820078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] PATH_INFO: `/v3/users/ffd06e916db54c61884c6f6aae64ddc0` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:46.821565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffd06e916db54c61884c6f6aae64ddc0)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:46.823526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:46.889497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffd06e916db54c61884c6f6aae64ddc0'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:46.890271 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffd06e916db54c61884c6f6aae64ddc0'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:46.905040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffd06e916db54c61884c6f6aae64ddc0'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:46.915288 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffd06e916db54c61884c6f6aae64ddc0'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:46.916007 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffd06e916db54c61884c6f6aae64ddc0'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:46.933354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1254/2527] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:46 2022] DELETE /identity/v3/users/ffd06e916db54c61884c6f6aae64ddc0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:46.935340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:46.935820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:46.936031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:46.955011 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:46.956266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:47.024415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1274/2528] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:47.053109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:47.054845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:47.080752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:47.127032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bc880-f3db-4f9e-ab24-bfe452815448 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:47.131454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:47.132269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:47.133090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:47.133090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:47.143705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:47.145136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:47.157059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1255/2529] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:47.718511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:47.720648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:47.740913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba543cd-8e12-47d4-8e40-eac4bc1b7ea8 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:47.754355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:47.755381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:47.755728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:47.756107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] PATH_INFO: `/v3/projects/9b5dab8d4a5e4feda1b3dd49375460ca` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:47.757602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5dab8d4a5e4feda1b3dd49375460ca)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:47.759824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:47.830166 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5dab8d4a5e4feda1b3dd49375460ca', 'request_id': 'req-34364117-3d33-4f3b-85c7-5c385e89d5b4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:47.838580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5dab8d4a5e4feda1b3dd49375460ca', 'request_id': 'req-34364117-3d33-4f3b-85c7-5c385e89d5b4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:47.864327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5dab8d4a5e4feda1b3dd49375460ca', 'request_id': 'req-34364117-3d33-4f3b-85c7-5c385e89d5b4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:47.864327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:47.887336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] The token cache is being invalidate because project 9b5dab8d4a5e4feda1b3dd49375460ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:47.887923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:47.891006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:47.896068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1275/2530] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:47 2022] DELETE /identity/v3/projects/9b5dab8d4a5e4feda1b3dd49375460ca => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:47.900179 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:47.911622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd240c-45e7-443d-bfc6-3c1e88335da5 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:47.995860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:47.997146 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:47.997603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:47.998036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] PATH_INFO: `/v3/users/967b776969c84b47bb94b705eb688ea1` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:48.000346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=967b776969c84b47bb94b705eb688ea1)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:48.003184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:48.094439 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '967b776969c84b47bb94b705eb688ea1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.095236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '967b776969c84b47bb94b705eb688ea1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.115586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '967b776969c84b47bb94b705eb688ea1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.127101 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '967b776969c84b47bb94b705eb688ea1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.127848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d77e4a47-ddd9-448f-8b00-ea3ed1b8f8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '967b776969c84b47bb94b705eb688ea1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.146486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1256/2531] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:47 2022] DELETE /identity/v3/users/967b776969c84b47bb94b705eb688ea1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:48.520237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:48.522423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:48.538265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34364117-3d33-4f3b-85c7-5c385e89d5b4 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:48.620164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:48.621048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:48.621321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:48.621589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] PATH_INFO: `/v3/projects/736a2fd18780437eaaf1f512949dc73c` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:48.622851 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=736a2fd18780437eaaf1f512949dc73c)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:48.625117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:48.695898 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736a2fd18780437eaaf1f512949dc73c', 'request_id': 'req-e77f4f59-8204-4d90-b33a-ef5ea908f935'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.704982 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736a2fd18780437eaaf1f512949dc73c', 'request_id': 'req-e77f4f59-8204-4d90-b33a-ef5ea908f935'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.715140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736a2fd18780437eaaf1f512949dc73c', 'request_id': 'req-e77f4f59-8204-4d90-b33a-ef5ea908f935'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.731727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] The token cache is being invalidate because project 736a2fd18780437eaaf1f512949dc73c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:48.732315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.733082 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-e77f4f59-8204-4d90-b33a-ef5ea908f935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:48.735326 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1276/2532] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:48 2022] DELETE /identity/v3/projects/736a2fd18780437eaaf1f512949dc73c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:48.740191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d54e-33b4-4a54-a951-d337db41d210 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:48.740484 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d54e-33b4-4a54-a951-d337db41d210 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:48.740766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129d54e-33b4-4a54-a951-d337db41d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:48.760872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c129d54e-33b4-4a54-a951-d337db41d210 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:48.762028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c129d54e-33b4-4a54-a951-d337db41d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:48.810306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:48.855232 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91e052-319d-441c-8eed-0f16d6331719 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:48.855632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91e052-319d-441c-8eed-0f16d6331719 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:48.855962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91e052-319d-441c-8eed-0f16d6331719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:48.858666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1257/2533] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:48.889829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-ab91e052-319d-441c-8eed-0f16d6331719 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:48.891382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ab91e052-319d-441c-8eed-0f16d6331719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:48.926021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1277/2534] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:48.930256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129d54e-33b4-4a54-a951-d337db41d210 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:48.942187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129d54e-33b4-4a54-a951-d337db41d210 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:48.949712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:48.951866 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:48.954329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:48.954804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:48.955719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:56:48.957228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:48.958872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:48.989725 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1258/2535] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:56:48.991295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91e052-319d-441c-8eed-0f16d6331719 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.003602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91e052-319d-441c-8eed-0f16d6331719 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.018435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.019225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.019626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.020016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.020922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.023645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.081958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1278/2536] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:48 2022] POST /identity/v3/projects => generated 391 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:49.083264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.095106 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e78757-8eb1-497b-94fb-9b0ac67e2f77 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.167731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.168560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.169571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.169571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.170208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.172385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.208463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1259/2537] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:49 2022] POST /identity/v3/users => generated 296 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:49.209266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.220814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d18b5d-2789-4908-aad8-64ad56fd2748 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.225935 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.227020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.227445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.227722 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.228497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.230137 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.244551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1279/2538] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:56:49.248200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.259445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04139a0-a4da-4f1b-970e-a4ae6e942b36 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.266901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643efd3-7d84-4a5d-92d2-e16243125ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.267549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643efd3-7d84-4a5d-92d2-e16243125ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.267833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643efd3-7d84-4a5d-92d2-e16243125ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.268151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643efd3-7d84-4a5d-92d2-e16243125ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.271380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643efd3-7d84-4a5d-92d2-e16243125ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.276732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643efd3-7d84-4a5d-92d2-e16243125ac4 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.291573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1260/2539] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:49.292780 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.303827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d002b-d512-4ef6-be0e-6ad047222247 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.315191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.316171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.316648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.317108 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] PATH_INFO: `/v3/projects/1add08ee4872467ba2041a70162d231b/users/6764cf272e5448f4a0406400745b18d7/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.341689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1add08ee4872467ba2041a70162d231b, user_id=6764cf272e5448f4a0406400745b18d7, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.345411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.374406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1280/2540] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:49 2022] PUT /identity/v3/projects/1add08ee4872467ba2041a70162d231b/users/6764cf272e5448f4a0406400745b18d7/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:49.386784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840737b4-5ea4-41a3-8935-b2f0f270bf87 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.387135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840737b4-5ea4-41a3-8935-b2f0f270bf87 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.387349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840737b4-5ea4-41a3-8935-b2f0f270bf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.422021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-840737b4-5ea4-41a3-8935-b2f0f270bf87 None None] MFA Rules not processed for user `6764cf272e5448f4a0406400745b18d7`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:49.423605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-840737b4-5ea4-41a3-8935-b2f0f270bf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:49.516520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1261/2541] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:49.522263 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.541925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:49.623076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd447531-eda1-4891-b0ea-a5f8631aebea admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.629649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.631325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.631772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.632909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.633060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.637401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.674036 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1281/2542] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:49 2022] POST /identity/v3/projects => generated 393 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:49.675873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840737b4-5ea4-41a3-8935-b2f0f270bf87 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.692326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840737b4-5ea4-41a3-8935-b2f0f270bf87 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.730092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:49.755646 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.756385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.756744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.757026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.757766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.759759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.785159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1262/2543] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:49 2022] POST /identity/v3/users => generated 303 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:49.789286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.800878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f4af8-0df0-40e9-8899-5c0fa4af9319 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.806276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.806981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.807253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.807526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.808300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.809927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.822119 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1282/2544] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:49.826407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.837762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa88c25-9e65-4bad-9b62-a4ed481e90d6 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.842407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.843314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.843629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.843907 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] PATH_INFO: `/v3/projects/4a60b0e6085a40d7827214d9f357edaa/users/4ee706336499488f9ad5d85b9d77633e/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.861543 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a60b0e6085a40d7827214d9f357edaa, user_id=4ee706336499488f9ad5d85b9d77633e, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.864266 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.886596 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1263/2545] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:49 2022] PUT /identity/v3/projects/4a60b0e6085a40d7827214d9f357edaa/users/4ee706336499488f9ad5d85b9d77633e/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:49.890912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:49.902970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6664c-4f07-4e1e-8ce5-367fa1588066 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:49.969875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbbdca-5564-4f4a-9527-62830b37d8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:49.970612 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbbdca-5564-4f4a-9527-62830b37d8dd admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:49.970888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbbdca-5564-4f4a-9527-62830b37d8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:49.971160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbbdca-5564-4f4a-9527-62830b37d8dd admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:49.971892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbbdca-5564-4f4a-9527-62830b37d8dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:49.973581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbbdca-5564-4f4a-9527-62830b37d8dd admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:49.990074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1283/2546] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:49.991080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:50.003229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8d65-af4b-487c-ada9-ddba27c6d556 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:50.008538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:50.009246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:50.009527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:50.009806 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] PATH_INFO: `/v3/projects/4a60b0e6085a40d7827214d9f357edaa/users/4ee706336499488f9ad5d85b9d77633e/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:50.012187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a60b0e6085a40d7827214d9f357edaa, user_id=4ee706336499488f9ad5d85b9d77633e, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:50.014990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:50.027024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1264/2547] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:49 2022] PUT /identity/v3/projects/4a60b0e6085a40d7827214d9f357edaa/users/4ee706336499488f9ad5d85b9d77633e/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:50.038461 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:50.038852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:50.039549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:50.072165 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] MFA Rules not processed for user `4ee706336499488f9ad5d85b9d77633e`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:50.073579 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:50.159009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1284/2548] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:50.171804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:50.173755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:50.234721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37984a91-c0d8-4fde-893a-8f18baa7d305 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:50.238391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:50.239047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:50.239330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:50.239605 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:50.249541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:50.251071 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:50.262211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1265/2549] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:50.316688 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:50.318515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:50.326159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:50.332590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965f47c-f623-4a3f-ba95-6cb68a40de5a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:50.338593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc2484-a83f-4cbc-982e-eb2a88022ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:50.339412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc2484-a83f-4cbc-982e-eb2a88022ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:50.339755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc2484-a83f-4cbc-982e-eb2a88022ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:50.340054 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc2484-a83f-4cbc-982e-eb2a88022ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:50.422844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc2484-a83f-4cbc-982e-eb2a88022ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:50.424517 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc2484-a83f-4cbc-982e-eb2a88022ae7 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:50.434416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1285/2550] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:53.974895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:54.059859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc61eb1-c5e6-43d7-9049-64afbe78cee9 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:54.067455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:54.068418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:54.068705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:54.069057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] PATH_INFO: `/v3/users/6764cf272e5448f4a0406400745b18d7` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:54.070469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6764cf272e5448f4a0406400745b18d7)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:54.072807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:54.123262 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:54.135137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6764cf272e5448f4a0406400745b18d7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:54.136108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6764cf272e5448f4a0406400745b18d7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:54.159826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6764cf272e5448f4a0406400745b18d7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:54.171280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6764cf272e5448f4a0406400745b18d7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:54.172193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6764cf272e5448f4a0406400745b18d7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:54.189850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1266/2551] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:53 2022] DELETE /identity/v3/users/6764cf272e5448f4a0406400745b18d7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:54.191576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:54.192057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:54.192229 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:54.212346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:54.213610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:54.278762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1286/2552] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:54.289238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:54.290689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:54.350421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8cea2-82c2-402b-b8df-12e0ea96733b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:54.354617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:54.355341 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:54.355611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:54.356121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:54.366541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:54.368368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:54.383600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1267/2553] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:56:54.953654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:54.955460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:54.984123 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10dfd1-7e35-4550-9948-cbc25352893d None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:54.994601 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:54.995664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:54.995764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:54.996106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] PATH_INFO: `/v3/projects/1add08ee4872467ba2041a70162d231b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:54.997955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=1add08ee4872467ba2041a70162d231b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:55.000093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:55.138149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1add08ee4872467ba2041a70162d231b', 'request_id': 'req-1259c76c-0582-4e74-9522-3328634da6de'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.156328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1add08ee4872467ba2041a70162d231b', 'request_id': 'req-1259c76c-0582-4e74-9522-3328634da6de'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.178481 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1add08ee4872467ba2041a70162d231b', 'request_id': 'req-1259c76c-0582-4e74-9522-3328634da6de'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.197419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] The token cache is being invalidate because project 1add08ee4872467ba2041a70162d231b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:55.198812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.198812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.200444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1287/2554] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:54 2022] DELETE /identity/v3/projects/1add08ee4872467ba2041a70162d231b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:55.204863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:55.217286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ab779-a288-47c1-8494-3a2333f0d6cd service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:55.338391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:55.340226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:55.340507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:55.340837 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] PATH_INFO: `/v3/users/4ee706336499488f9ad5d85b9d77633e` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:55.343065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee706336499488f9ad5d85b9d77633e)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:55.345081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:55.437537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee706336499488f9ad5d85b9d77633e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.438272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee706336499488f9ad5d85b9d77633e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.449441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:55.461464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee706336499488f9ad5d85b9d77633e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.474047 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee706336499488f9ad5d85b9d77633e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.475178 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-de93013b-acb4-4875-a4bb-1667ee6190f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee706336499488f9ad5d85b9d77633e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:55.497368 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1268/2555] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:55 2022] DELETE /identity/v3/users/4ee706336499488f9ad5d85b9d77633e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:56.308619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:56.310171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:56.331130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259c76c-0582-4e74-9522-3328634da6de admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:56.356038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:56.414198 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:56.415121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:56.415804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:56.415804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] PATH_INFO: `/v3/projects/4a60b0e6085a40d7827214d9f357edaa` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:56.417240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a60b0e6085a40d7827214d9f357edaa)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:56.419173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:56.477423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a60b0e6085a40d7827214d9f357edaa', 'request_id': 'req-80b333a4-379a-49a0-9ee9-aed52ab69513'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:56.485096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a60b0e6085a40d7827214d9f357edaa', 'request_id': 'req-80b333a4-379a-49a0-9ee9-aed52ab69513'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:56.504266 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a60b0e6085a40d7827214d9f357edaa', 'request_id': 'req-80b333a4-379a-49a0-9ee9-aed52ab69513'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:56.520181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] The token cache is being invalidate because project 4a60b0e6085a40d7827214d9f357edaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:56.520476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:56.521141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-80b333a4-379a-49a0-9ee9-aed52ab69513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:56.524116 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1288/2556] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:56 2022] DELETE /identity/v3/projects/4a60b0e6085a40d7827214d9f357edaa => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:56.528940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863f791-0785-4546-9342-574b433e88bf None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:56.529433 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863f791-0785-4546-9342-574b433e88bf None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:56.529532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863f791-0785-4546-9342-574b433e88bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:56.555256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6863f791-0785-4546-9342-574b433e88bf None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:56.556680 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6863f791-0785-4546-9342-574b433e88bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:56.718586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1269/2557] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:56.720922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d1f07-7883-41ab-a5bf-6d40996b209a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:56.721342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d1f07-7883-41ab-a5bf-6d40996b209a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:56.721625 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d1f07-7883-41ab-a5bf-6d40996b209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:56.754749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-0d2d1f07-7883-41ab-a5bf-6d40996b209a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:56.756288 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-0d2d1f07-7883-41ab-a5bf-6d40996b209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:56.774424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1289/2558] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:56.779259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863f791-0785-4546-9342-574b433e88bf None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:56.806713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863f791-0785-4546-9342-574b433e88bf None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:56.812959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:56.814297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:56.814453 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:56.814971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:56.817893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:56.818595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:56.819052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:56.824760 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:56:56.825287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:56.827148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:56.848670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:56.850631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:56.853732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1270/2559] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:56:56.869822 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:56.886289 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff905-c435-42aa-b138-e6ea0ec4987e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:56.893059 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:56.894238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:56.894643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:56.895014 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:56.895874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:56.898520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:56.953370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1271/2560] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:56 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:56:56.962679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1290/2561] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:56.971078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:56.979037 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:56.979637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:57.075180 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:57.106193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677efb67-b853-4eee-854c-abc2a42c3dc2 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.126944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c3473-2931-4029-9047-d6f5e5cce0d3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.134832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.136461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.136935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.137354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.138559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.143152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.174904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1272/2562] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:56 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:57.185534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:57.196082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ba249-0be0-4b10-bf8d-e02060a5f8ee admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.198006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b97d6-8a0e-4feb-be92-45c4adfa22fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.198827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b97d6-8a0e-4feb-be92-45c4adfa22fa service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.198964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b97d6-8a0e-4feb-be92-45c4adfa22fa service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.199212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b97d6-8a0e-4feb-be92-45c4adfa22fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.201121 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.201785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.202066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.202338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.203122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.204731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.216562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1273/2563] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:57.220793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:57.233567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a8ef5-5bff-40c5-bd45-b458cdd9f72b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.242159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.242859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.243135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.243403 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.244270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.245910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.257911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1274/2564] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:57.261868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:57.262318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b97d6-8a0e-4feb-be92-45c4adfa22fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.263972 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b97d6-8a0e-4feb-be92-45c4adfa22fa service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.278461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b214c3-f774-42f2-b915-9d10069f3218 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.281395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1291/2565] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:56:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:56:57.289317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.290232 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.290512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.290785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] PATH_INFO: `/v3/projects/e821aeb320cc48389e0f295f203520f0/users/7c466773d19b4e999f02562390b9c50b/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.293727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:57.311772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e821aeb320cc48389e0f295f203520f0, user_id=7c466773d19b4e999f02562390b9c50b, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.315537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.340050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1275/2566] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:57 2022] PUT /identity/v3/projects/e821aeb320cc48389e0f295f203520f0/users/7c466773d19b4e999f02562390b9c50b/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:57.353181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.354719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.355219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.392983 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] MFA Rules not processed for user `7c466773d19b4e999f02562390b9c50b`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:57.394464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:57.484106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1292/2567] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:57.491961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:57.493019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:57.496799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:57.498389 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:57.533958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:57.597178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6424a-a06e-4d47-8e4b-e85ae7871259 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.605060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.605695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.605979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.606374 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.617173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.619236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.622006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b20436-24c5-4b15-b115-136b6c90b0b5 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.629256 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.629934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.630303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.630582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.632715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:57.644474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1293/2568] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:56:57.646366 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:57.649812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:57.655291 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.657704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.670049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1276/2569] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:57.671285 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:57.778239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b091b65-6f0c-4358-bead-1486fa619a07 service placement] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.780734 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99800460-70bd-4153-9543-2cb9ded89a69 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:57.787086 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.788174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.788457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.788859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] PATH_INFO: `/v3/users/7c466773d19b4e999f02562390b9c50b` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.790174 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c466773d19b4e999f02562390b9c50b)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.790697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:57.791681 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.792158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.792270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.792531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] PATH_INFO: `/v3/users/b8203a321cce4c558a69ff6ca75a4d27` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.794093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8203a321cce4c558a69ff6ca75a4d27)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:57.796230 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:57.863522 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c466773d19b4e999f02562390b9c50b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.864275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c466773d19b4e999f02562390b9c50b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.866568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8203a321cce4c558a69ff6ca75a4d27'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.880626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c466773d19b4e999f02562390b9c50b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.891434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8203a321cce4c558a69ff6ca75a4d27'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.896206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c466773d19b4e999f02562390b9c50b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.897102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bdb18d60-aa84-4202-9fc3-1b103ffabc0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c466773d19b4e999f02562390b9c50b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.899848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:57.901997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8203a321cce4c558a69ff6ca75a4d27'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.903021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8203a321cce4c558a69ff6ca75a4d27'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.905098 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8203a321cce4c558a69ff6ca75a4d27'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:57.914421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1277/2570] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:57 2022] DELETE /identity/v3/users/7c466773d19b4e999f02562390b9c50b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:57.919889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:57.919889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:57.920523 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:57.934009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1294/2571] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:57 2022] DELETE /identity/v3/users/b8203a321cce4c558a69ff6ca75a4d27 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:57.941937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:57.943345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:58.030516 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1278/2572] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:56:58.040852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:58.042511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:58.119485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:58.124097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd05a6-a90b-49e6-8036-995a27366313 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:58.128511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:58.130728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:58.131007 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:58.131520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:58.148107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:58.149596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:58.164024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1295/2573] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:56:58.558075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:58.559829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:58.579611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f5316-b4cb-4c69-9479-f8702afae8f5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:58.601265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:58.602212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:58.602502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:58.602784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] PATH_INFO: `/v3/projects/e1f6fecd9b364f3bbc6281bf3a2e8ade` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:58.604272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f6fecd9b364f3bbc6281bf3a2e8ade)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:58.606233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:58.628276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:58.629714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:58.648565 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c7ca3-c8bc-41f3-900c-7a5e289cd0ff service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:58.664699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:58.665243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:58.665526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:58.665800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] PATH_INFO: `/v3/projects/e821aeb320cc48389e0f295f203520f0` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:58.668567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e821aeb320cc48389e0f295f203520f0)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:58.670532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:58.709532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f6fecd9b364f3bbc6281bf3a2e8ade', 'request_id': 'req-cf145eb6-1093-46fb-9bd7-8574d97f38bf'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.721801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f6fecd9b364f3bbc6281bf3a2e8ade', 'request_id': 'req-cf145eb6-1093-46fb-9bd7-8574d97f38bf'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.731372 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f6fecd9b364f3bbc6281bf3a2e8ade', 'request_id': 'req-cf145eb6-1093-46fb-9bd7-8574d97f38bf'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.766562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] The token cache is being invalidate because project e1f6fecd9b364f3bbc6281bf3a2e8ade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:58.766953 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.768708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.773258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e821aeb320cc48389e0f295f203520f0', 'request_id': 'req-5ba0a3dc-fca4-4f61-903d-8685d3778da1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.777340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1279/2574] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:58 2022] DELETE /identity/v3/projects/e1f6fecd9b364f3bbc6281bf3a2e8ade => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:58.778493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:58.780985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:58.788775 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e821aeb320cc48389e0f295f203520f0', 'request_id': 'req-5ba0a3dc-fca4-4f61-903d-8685d3778da1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.807595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e821aeb320cc48389e0f295f203520f0', 'request_id': 'req-5ba0a3dc-fca4-4f61-903d-8685d3778da1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.829317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] The token cache is being invalidate because project e821aeb320cc48389e0f295f203520f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:58.829643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.832524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-5ba0a3dc-fca4-4f61-903d-8685d3778da1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:58.834665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1296/2575] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:58 2022] DELETE /identity/v3/projects/e821aeb320cc48389e0f295f203520f0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:56:58.841813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cda193-90fb-4c0b-8801-1f7dea25a0ce None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:58.842124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cda193-90fb-4c0b-8801-1f7dea25a0ce None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:58.846324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cda193-90fb-4c0b-8801-1f7dea25a0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:58.890218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-96cda193-90fb-4c0b-8801-1f7dea25a0ce None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:58.891481 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-96cda193-90fb-4c0b-8801-1f7dea25a0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:58.934985 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf145eb6-1093-46fb-9bd7-8574d97f38bf admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:58.969335 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:59.001505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1297/2576] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:59.003049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39588c-3219-43c2-924b-35593958e289 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.003487 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39588c-3219-43c2-924b-35593958e289 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.003773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39588c-3219-43c2-924b-35593958e289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.017789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.019359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.019804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.020362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] PATH_INFO: `/v3/users/fe3e3046e053405fa43f401688c5aa02` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.022925 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3e3046e053405fa43f401688c5aa02)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.024513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-4a39588c-3219-43c2-924b-35593958e289 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:59.025074 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.025758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-4a39588c-3219-43c2-924b-35593958e289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:59.043451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1298/2577] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:59.047941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39588c-3219-43c2-924b-35593958e289 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.059125 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39588c-3219-43c2-924b-35593958e289 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:59.065088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.065883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.066251 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.066511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.067296 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:56:59.067592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.072219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.092044 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1299/2578] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:56:59.095538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.097500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3e3046e053405fa43f401688c5aa02'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.098195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3e3046e053405fa43f401688c5aa02'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.109669 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ef753-048b-4da7-bcc6-a556a6dd981f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:59.113793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3e3046e053405fa43f401688c5aa02'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.117908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.119222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.119613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.121864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.123192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.125190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.132535 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3e3046e053405fa43f401688c5aa02'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.133554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3e3046e053405fa43f401688c5aa02'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.151357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1280/2579] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:56:58 2022] DELETE /identity/v3/users/fe3e3046e053405fa43f401688c5aa02 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:59.176877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1300/2580] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:56:59 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:56:59.180312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.181970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:59.284690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdf1a2f-0f3d-4441-a921-f4361dcbccee admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:59.367292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:59.391398 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.392370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.392603 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.393792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.394599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.397947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.412883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.416170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:59.431637 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e188f6-14ce-486b-a2a2-a0088b5d3524 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:59.436614 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1281/2581] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:56:59 2022] POST /identity/v3/users => generated 289 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:56:59.436763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.437591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.437877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.438152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] PATH_INFO: `/v3/projects/d74bc29e60ff431fb5536dcddccd39cd` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.438261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.439368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74bc29e60ff431fb5536dcddccd39cd)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.441819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.453434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce70e3-49af-4270-ae58-8eaedb62a610 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:59.460277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.461031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.461320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.464329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.464329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.464329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.477666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1282/2582] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:59.482692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.493474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:56:59.496211 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8784-643a-4b0e-8b89-1b9f8cdd9269 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:59.501450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.502361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.502664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.502942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.503790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.505499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.518428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74bc29e60ff431fb5536dcddccd39cd', 'request_id': 'req-0165f94e-c281-4f51-9933-ed363c60b52b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.519382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1283/2583] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:56:59.523287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.529303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74bc29e60ff431fb5536dcddccd39cd', 'request_id': 'req-0165f94e-c281-4f51-9933-ed363c60b52b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.540176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ddf39-78e5-4382-a0fb-0c837f38be0c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:56:59.544520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74bc29e60ff431fb5536dcddccd39cd', 'request_id': 'req-0165f94e-c281-4f51-9933-ed363c60b52b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.557886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] The token cache is being invalidate because project d74bc29e60ff431fb5536dcddccd39cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:56:59.558341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.559157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-0165f94e-c281-4f51-9933-ed363c60b52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:56:59.561020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1301/2584] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:56:59 2022] DELETE /identity/v3/projects/d74bc29e60ff431fb5536dcddccd39cd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:59.643663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:56:59.645144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.645588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.646019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] PATH_INFO: `/v3/projects/b7a815cfdd604f4e9e897f8283243481/users/4a47a7fe4714468ab69f9eb65ca03677/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.667443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a815cfdd604f4e9e897f8283243481, user_id=4a47a7fe4714468ab69f9eb65ca03677, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:56:59.671041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:56:59.698644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1284/2585] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:56:59 2022] PUT /identity/v3/projects/b7a815cfdd604f4e9e897f8283243481/users/4a47a7fe4714468ab69f9eb65ca03677/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:56:59.713358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:56:59.713955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:56:59.714159 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:56:59.755878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] MFA Rules not processed for user `4a47a7fe4714468ab69f9eb65ca03677`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:56:59.758074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:56:59.868643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1302/2586] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:56:59.881226 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:56:59.883402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:00.002788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc225577-d618-44be-8829-a14ca5efb2c3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:00.012325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191bc05-232f-4c17-936f-2cdf115ea951 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:00.013088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191bc05-232f-4c17-936f-2cdf115ea951 service glance] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:00.013367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191bc05-232f-4c17-936f-2cdf115ea951 service glance] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:00.013640 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191bc05-232f-4c17-936f-2cdf115ea951 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:00.023890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191bc05-232f-4c17-936f-2cdf115ea951 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:00.025827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191bc05-232f-4c17-936f-2cdf115ea951 service glance] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:00.044119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1285/2587] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:56:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:09.785070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:09.786643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:09.849787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c99e8-75b0-45db-96d9-6fed56323f87 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:09.854709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af365e-1960-495b-9c7c-ce983b4a3bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:09.855357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af365e-1960-495b-9c7c-ce983b4a3bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:09.855633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af365e-1960-495b-9c7c-ce983b4a3bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:09.855900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af365e-1960-495b-9c7c-ce983b4a3bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:09.857997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-28af365e-1960-495b-9c7c-ce983b4a3bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:09.928386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:09.931261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af365e-1960-495b-9c7c-ce983b4a3bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:09.933412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af365e-1960-495b-9c7c-ce983b4a3bd4 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:09.946857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1303/2588] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:10.033294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=82,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:13.120569 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5d6fe-287d-4031-ada4-7efc503519db None None] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:13.121689 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5d6fe-287d-4031-ada4-7efc503519db None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:13.122155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5d6fe-287d-4031-ada4-7efc503519db None None] PATH_INFO: `/` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:13.124021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1286/2589] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:57:13.131303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b558b-d154-4c26-84e2-6cff781b2bb0 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:13.131856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b558b-d154-4c26-84e2-6cff781b2bb0 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:13.131856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b558b-d154-4c26-84e2-6cff781b2bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:13.154003 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-3c5b558b-d154-4c26-84e2-6cff781b2bb0 None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:13.155666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3c5b558b-d154-4c26-84e2-6cff781b2bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:13.232356 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1304/2590] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:13.245905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5d6fe-287d-4031-ada4-7efc503519db None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:13.248215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bab5d6fe-287d-4031-ada4-7efc503519db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:13.262686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5d6fe-287d-4031-ada4-7efc503519db None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:13.266199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d029ed2-ebf7-4e94-876a-e3bab58d9ea5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:13.266846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d029ed2-ebf7-4e94-876a-e3bab58d9ea5 service glance] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:13.267183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d029ed2-ebf7-4e94-876a-e3bab58d9ea5 service glance] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:13.267399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d029ed2-ebf7-4e94-876a-e3bab58d9ea5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:13.278255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d029ed2-ebf7-4e94-876a-e3bab58d9ea5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:13.279693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d029ed2-ebf7-4e94-876a-e3bab58d9ea5 service glance] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:13.291412 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1287/2591] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:13.361257 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f9755-8eba-4c6d-aa63-e21f3768a83c None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:13.361569 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f9755-8eba-4c6d-aa63-e21f3768a83c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:13.361840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f9755-8eba-4c6d-aa63-e21f3768a83c None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:13.363270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1305/2592] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:57:13.368593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e611d-a9bd-443c-86cd-76df22fa2f44 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:13.368883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e611d-a9bd-443c-86cd-76df22fa2f44 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:13.369164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e611d-a9bd-443c-86cd-76df22fa2f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:13.389663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d89e611d-a9bd-443c-86cd-76df22fa2f44 None None] MFA Rules not processed for user `5e17d25a83fe48d99ddbd209fbea8486`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:13.390877 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d89e611d-a9bd-443c-86cd-76df22fa2f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:13.397759 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1288/2593] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:57:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:57:13.409993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f9755-8eba-4c6d-aa63-e21f3768a83c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:13.423878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f9755-8eba-4c6d-aa63-e21f3768a83c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:13.427733 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:13.428394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:13.428672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:13.429004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:13.436995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:13.438557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:13.447071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1306/2594] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:16.179938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe228153-288c-4c85-8801-699998be8849 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:16.180443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe228153-288c-4c85-8801-699998be8849 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:16.180711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe228153-288c-4c85-8801-699998be8849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:16.199949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-fe228153-288c-4c85-8801-699998be8849 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:16.201151 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fe228153-288c-4c85-8801-699998be8849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:16.257705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1289/2595] 10.222.0.50 () {58 vars in 1009 bytes} [Thu Jun 30 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:57:16.270209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:16.271691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:16.344491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7626b04-b8e7-416a-b646-8ed04a0b0cf1 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:16.350351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf2432-ead4-4dfe-8e57-e26b825ca090 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:16.351018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2432-ead4-4dfe-8e57-e26b825ca090 service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:16.351277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2432-ead4-4dfe-8e57-e26b825ca090 service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:16.351540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf2432-ead4-4dfe-8e57-e26b825ca090 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:16.360399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf2432-ead4-4dfe-8e57-e26b825ca090 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:16.361997 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cf2432-ead4-4dfe-8e57-e26b825ca090 service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:16.370840 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1307/2596] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:57:20.056789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe228153-288c-4c85-8801-699998be8849 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:20.058315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-fe228153-288c-4c85-8801-699998be8849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:20.127363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe228153-288c-4c85-8801-699998be8849 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:20.132051 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:20.132943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:20.133221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:20.133495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] PATH_INFO: `/v3/users/4a47a7fe4714468ab69f9eb65ca03677` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:20.134727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a47a7fe4714468ab69f9eb65ca03677)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:20.136719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:20.193769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a47a7fe4714468ab69f9eb65ca03677'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.194428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a47a7fe4714468ab69f9eb65ca03677'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.207799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a47a7fe4714468ab69f9eb65ca03677'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.217518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a47a7fe4714468ab69f9eb65ca03677'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.218319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a47a7fe4714468ab69f9eb65ca03677'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.232996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1290/2597] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:57:20 2022] DELETE /identity/v3/users/4a47a7fe4714468ab69f9eb65ca03677 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:20.237698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:20.238239 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:20.238360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:20.261318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:20.261318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:20.327934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1308/2598] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:20.338103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:20.339557 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:20.346429 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:20.407828 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ebd88-e42a-48f9-9ff6-44f63505c453 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:20.411914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee324959-2326-4b7a-8475-fd77d294f5d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:20.412712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee324959-2326-4b7a-8475-fd77d294f5d9 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:20.413094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee324959-2326-4b7a-8475-fd77d294f5d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:20.413468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee324959-2326-4b7a-8475-fd77d294f5d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:20.423731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee324959-2326-4b7a-8475-fd77d294f5d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:20.425264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee324959-2326-4b7a-8475-fd77d294f5d9 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:20.435941 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1291/2599] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:20.737477 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:20.739202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:20.754699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=100 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:20.759814 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743bf51d-8c8f-4241-b188-98629586f330 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:20.774653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:20.775703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:20.776003 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:20.776283 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] PATH_INFO: `/v3/projects/b7a815cfdd604f4e9e897f8283243481` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:20.785276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a815cfdd604f4e9e897f8283243481)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:20.787467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:20.841657 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a815cfdd604f4e9e897f8283243481', 'request_id': 'req-6bb157c0-af42-4182-a22d-9c7e3bff97bc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.848459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a815cfdd604f4e9e897f8283243481', 'request_id': 'req-6bb157c0-af42-4182-a22d-9c7e3bff97bc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.859270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a815cfdd604f4e9e897f8283243481', 'request_id': 'req-6bb157c0-af42-4182-a22d-9c7e3bff97bc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.871110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] The token cache is being invalidate because project b7a815cfdd604f4e9e897f8283243481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:57:20.871512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.872308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-6bb157c0-af42-4182-a22d-9c7e3bff97bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:20.877714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1309/2600] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:57:20 2022] DELETE /identity/v3/projects/b7a815cfdd604f4e9e897f8283243481 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:20.881868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b808761-7094-4d63-8946-e94ea0d88f73 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:20.882085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b808761-7094-4d63-8946-e94ea0d88f73 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:20.882369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b808761-7094-4d63-8946-e94ea0d88f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:20.901731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8b808761-7094-4d63-8946-e94ea0d88f73 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:20.903333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8b808761-7094-4d63-8946-e94ea0d88f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:20.991200 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1292/2601] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:20.993048 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c77123-4cdc-4d54-bafc-dba6b6cbf4d1 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:20.993375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c77123-4cdc-4d54-bafc-dba6b6cbf4d1 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:20.993665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c77123-4cdc-4d54-bafc-dba6b6cbf4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.013438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-20c77123-4cdc-4d54-bafc-dba6b6cbf4d1 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:21.014661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-20c77123-4cdc-4d54-bafc-dba6b6cbf4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:21.022848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1310/2602] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:21.027456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b808761-7094-4d63-8946-e94ea0d88f73 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.037858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b808761-7094-4d63-8946-e94ea0d88f73 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.042633 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.043318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.043604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.043884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.044664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:57:21.044974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.046618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.063113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1293/2603] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:57:21.065171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c77123-4cdc-4d54-bafc-dba6b6cbf4d1 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.075181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c77123-4cdc-4d54-bafc-dba6b6cbf4d1 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.079830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.080596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.080880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.081150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.081849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.083815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.111981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1311/2604] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:57:21 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:57:21.116275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.127348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5d95c-acb4-472b-8a0b-f4045b42bdfc admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.194181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.194893 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.195247 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.195594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.196413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.198442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.224039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1294/2605] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:57:21 2022] POST /identity/v3/users => generated 281 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:57:21.228511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.239721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4501e511-0df3-4456-87d3-6bdd9cf45817 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.244725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.245492 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.245878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.246217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.247022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.248760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.263686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1312/2606] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:21.264580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.275978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2906dc3-9263-4adf-ad70-2b5d8f9cac30 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.281534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48784f26-8dba-4154-85d7-89201b26aa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.282213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48784f26-8dba-4154-85d7-89201b26aa65 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.282494 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48784f26-8dba-4154-85d7-89201b26aa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.282797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48784f26-8dba-4154-85d7-89201b26aa65 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.283514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48784f26-8dba-4154-85d7-89201b26aa65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.285214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48784f26-8dba-4154-85d7-89201b26aa65 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.296461 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1295/2607] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:21.300713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.311631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffc448-93b4-48b2-bb40-8efb8173c3d7 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.317302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.318444 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.318879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.319221 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] PATH_INFO: `/v3/projects/6c0c0b432b654bc497c182f7841400ed/users/3494fbfde0b94fb3a17ac8d51cb4aa3a/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.335847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0c0b432b654bc497c182f7841400ed, user_id=3494fbfde0b94fb3a17ac8d51cb4aa3a, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.338782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.357358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1313/2608] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:57:21 2022] PUT /identity/v3/projects/6c0c0b432b654bc497c182f7841400ed/users/3494fbfde0b94fb3a17ac8d51cb4aa3a/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:21.430422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c94489-c62b-4eff-9ae8-3a0f49292111 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.431082 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c94489-c62b-4eff-9ae8-3a0f49292111 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.431527 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c94489-c62b-4eff-9ae8-3a0f49292111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.458979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:21.462264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-25c94489-c62b-4eff-9ae8-3a0f49292111 None None] MFA Rules not processed for user `3494fbfde0b94fb3a17ac8d51cb4aa3a`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:21.465189 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-25c94489-c62b-4eff-9ae8-3a0f49292111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:21.536150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1296/2609] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:21.584051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.585932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:21.650822 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e141a-7f3b-43d1-b05a-ddd7e58e2af1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.654488 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2d40a-8dc2-465a-afba-6de51db57da1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.655141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d40a-8dc2-465a-afba-6de51db57da1 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.655420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d40a-8dc2-465a-afba-6de51db57da1 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.655692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2d40a-8dc2-465a-afba-6de51db57da1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.666826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2d40a-8dc2-465a-afba-6de51db57da1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.668369 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2d40a-8dc2-465a-afba-6de51db57da1 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.677980 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1314/2610] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:21.694019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c94489-c62b-4eff-9ae8-3a0f49292111 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:21.772724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c94489-c62b-4eff-9ae8-3a0f49292111 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:21.778010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:21.779000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.779347 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.779744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] PATH_INFO: `/v3/users/3494fbfde0b94fb3a17ac8d51cb4aa3a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.781040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3494fbfde0b94fb3a17ac8d51cb4aa3a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:21.783069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:21.839006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3494fbfde0b94fb3a17ac8d51cb4aa3a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:21.839712 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3494fbfde0b94fb3a17ac8d51cb4aa3a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:21.860563 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3494fbfde0b94fb3a17ac8d51cb4aa3a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:21.876135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3494fbfde0b94fb3a17ac8d51cb4aa3a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:21.876992 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3494fbfde0b94fb3a17ac8d51cb4aa3a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:21.916152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1297/2611] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:57:21 2022] DELETE /identity/v3/users/3494fbfde0b94fb3a17ac8d51cb4aa3a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:21.921475 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:21.921796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:21.922948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:21.957103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:21.958755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:21.995736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:22.094502 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1315/2612] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:22.105026 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:22.106711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:22.135648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:22.180763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c3c9d-0765-4e47-994b-ff715bd60c82 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:22.184649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc07729-57ef-4a42-9da9-2cfa26249c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:22.185384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc07729-57ef-4a42-9da9-2cfa26249c32 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:22.185732 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc07729-57ef-4a42-9da9-2cfa26249c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:22.186072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc07729-57ef-4a42-9da9-2cfa26249c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:22.201168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc07729-57ef-4a42-9da9-2cfa26249c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:22.202711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc07729-57ef-4a42-9da9-2cfa26249c32 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:22.213399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1298/2613] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:22.521664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:22.523877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:22.573391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6b2c3-b3f0-486c-864c-e3682e3a3e09 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:22.580586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:22.581736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:22.582193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:22.582474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] PATH_INFO: `/v3/projects/6c0c0b432b654bc497c182f7841400ed` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:22.583818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0c0b432b654bc497c182f7841400ed)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:22.585800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:22.645473 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0c0b432b654bc497c182f7841400ed', 'request_id': 'req-3655c6e3-8ed3-4f0e-917d-7171b90028fa'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:22.652084 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0c0b432b654bc497c182f7841400ed', 'request_id': 'req-3655c6e3-8ed3-4f0e-917d-7171b90028fa'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:22.662549 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0c0b432b654bc497c182f7841400ed', 'request_id': 'req-3655c6e3-8ed3-4f0e-917d-7171b90028fa'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:22.676375 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] The token cache is being invalidate because project 6c0c0b432b654bc497c182f7841400ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:57:22.676711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:22.677413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3655c6e3-8ed3-4f0e-917d-7171b90028fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:22.679515 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1316/2614] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:57:22 2022] DELETE /identity/v3/projects/6c0c0b432b654bc497c182f7841400ed => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:22.686793 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128411-f778-42b5-9f88-3391840bba24 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:22.687167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128411-f778-42b5-9f88-3391840bba24 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:22.687449 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128411-f778-42b5-9f88-3391840bba24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:22.705979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-7c128411-f778-42b5-9f88-3391840bba24 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:22.707294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-7c128411-f778-42b5-9f88-3391840bba24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:22.769652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1299/2615] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:22.778311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618eeb0-e399-4f08-a24e-fc946eee7539 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:22.778720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618eeb0-e399-4f08-a24e-fc946eee7539 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:22.778851 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4618eeb0-e399-4f08-a24e-fc946eee7539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:22.797900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-4618eeb0-e399-4f08-a24e-fc946eee7539 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:22.799129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-4618eeb0-e399-4f08-a24e-fc946eee7539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:22.809729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1317/2616] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:22.814090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c128411-f778-42b5-9f88-3391840bba24 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:22.828447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c128411-f778-42b5-9f88-3391840bba24 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:22.833351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:22.834040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:22.834317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:22.834610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:22.835375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:57:22.835686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:22.838002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:22.858349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1300/2617] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:57:22.859695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618eeb0-e399-4f08-a24e-fc946eee7539 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:22.869442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4618eeb0-e399-4f08-a24e-fc946eee7539 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:22.874357 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:22.875031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:22.875313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:22.875586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:22.876337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:22.878448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:22.907874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1318/2618] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:57:22 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:57:22.911850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:22.924337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491bcb0-a851-4d83-acf4-3836a1a8d8e3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:22.989974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:22.990836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:22.991125 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:22.991513 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:22.992261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:22.994208 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:23.020193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1301/2619] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:57:22 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:57:23.024402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:23.038328 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbda98-26c3-4c8e-bad5-a03a4f96e1a2 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:23.053860 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:23.054630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:23.055150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:23.055434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:23.056219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:23.058067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:23.075016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1319/2620] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:23.077140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:23.087904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150236f2-a3e9-4c28-96fb-a98a0d88559e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:23.095015 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ce6fb-51f5-4806-a138-355f884f7dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:23.095698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ce6fb-51f5-4806-a138-355f884f7dff admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:23.096010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ce6fb-51f5-4806-a138-355f884f7dff admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:23.096291 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ce6fb-51f5-4806-a138-355f884f7dff admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:23.097090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ce6fb-51f5-4806-a138-355f884f7dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:23.098726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ce6fb-51f5-4806-a138-355f884f7dff admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:23.114769 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1302/2621] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:23.115498 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:23.131370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca5b3d-ab3d-4cc6-b922-7898662c3643 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:23.138060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:23.139728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:23.140284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:23.140736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] PATH_INFO: `/v3/projects/4bf6984700f84c04a86fe0d7e4082c99/users/1c97ebbdf11e4fbf971856093fcc3dbe/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:23.162717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf6984700f84c04a86fe0d7e4082c99, user_id=1c97ebbdf11e4fbf971856093fcc3dbe, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:23.166566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:23.210800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1320/2622] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:57:23 2022] PUT /identity/v3/projects/4bf6984700f84c04a86fe0d7e4082c99/users/1c97ebbdf11e4fbf971856093fcc3dbe/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:57:23.227607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6e265-a90c-431e-b901-88c0b64fb9f8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:23.228081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6e265-a90c-431e-b901-88c0b64fb9f8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:23.228438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6e265-a90c-431e-b901-88c0b64fb9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:23.262531 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-63b6e265-a90c-431e-b901-88c0b64fb9f8 None None] MFA Rules not processed for user `1c97ebbdf11e4fbf971856093fcc3dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:23.264703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-63b6e265-a90c-431e-b901-88c0b64fb9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:23.292958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:23.356056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1303/2623] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:23.369223 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:23.371107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:23.434864 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:23.441272 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0048775-61e4-4d2b-a34e-78c5fb2db606 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:23.446831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:23.448167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:23.448588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:23.448928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:23.458449 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:23.460083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:23.471590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1321/2624] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:33.352567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=18 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:33.466778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:38.173310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fe4b-20cf-479d-900c-bfb1a7a0df94 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:38.174033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fe4b-20cf-479d-900c-bfb1a7a0df94 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:38.174746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c53fe4b-20cf-479d-900c-bfb1a7a0df94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:38.209996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-5c53fe4b-20cf-479d-900c-bfb1a7a0df94 None None] MFA Rules not processed for user `180e36ec0b254ae1813b2892c3117d87`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:38.211421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-5c53fe4b-20cf-479d-900c-bfb1a7a0df94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:38.310914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1304/2625] 10.222.0.50 () {58 vars in 967 bytes} [Thu Jun 30 15:57:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:38.325100 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:38.327748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:38.421993 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204915d1-30e2-42cf-8ca9-546741b7c659 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:38.429469 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fc6a4-9d34-4142-be9c-8f1033a14efb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2933f88027fc435eba3064124e2dd5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:38.430270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fc6a4-9d34-4142-be9c-8f1033a14efb service swift] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:38.430546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fc6a4-9d34-4142-be9c-8f1033a14efb service swift] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:38.430815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fc6a4-9d34-4142-be9c-8f1033a14efb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:38.439756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fc6a4-9d34-4142-be9c-8f1033a14efb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:38.445254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fc6a4-9d34-4142-be9c-8f1033a14efb service swift] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:38.455466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1322/2626] 10.222.0.50 () {60 vars in 1384 bytes} [Thu Jun 30 15:57:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 15:57:43.067854 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fe4b-20cf-479d-900c-bfb1a7a0df94 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:43.142020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c53fe4b-20cf-479d-900c-bfb1a7a0df94 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:43.147442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:43.151109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:43.151414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:43.151701 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] PATH_INFO: `/v3/users/1c97ebbdf11e4fbf971856093fcc3dbe` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:43.153165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c97ebbdf11e4fbf971856093fcc3dbe)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:43.155118 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:43.215486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c97ebbdf11e4fbf971856093fcc3dbe'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.216204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c97ebbdf11e4fbf971856093fcc3dbe'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.228434 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c97ebbdf11e4fbf971856093fcc3dbe'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.237249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c97ebbdf11e4fbf971856093fcc3dbe'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.237880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c97ebbdf11e4fbf971856093fcc3dbe'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.250153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1305/2627] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:57:43 2022] DELETE /identity/v3/users/1c97ebbdf11e4fbf971856093fcc3dbe => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:43.254519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:43.255007 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:43.255105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:43.272658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:43.273854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:43.334590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1323/2628] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:43.344478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:43.346201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:43.375273 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:43.406184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a078a50-19f1-4750-b83c-a4aa6e101aef admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:43.409938 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:43.410613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:43.410908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:43.411184 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:43.420212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:43.421658 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:43.431021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1306/2629] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:43.714167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:43.715820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:43.729289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e62439-79fe-42cf-8156-5fb45e3dd65f None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:43.734797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:43.735810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:43.739140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:43.739140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] PATH_INFO: `/v3/projects/4bf6984700f84c04a86fe0d7e4082c99` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:43.739430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf6984700f84c04a86fe0d7e4082c99)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:43.741354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:43.793304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf6984700f84c04a86fe0d7e4082c99', 'request_id': 'req-3b732afe-7c48-4c1e-8778-3022b17478cc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.799752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf6984700f84c04a86fe0d7e4082c99', 'request_id': 'req-3b732afe-7c48-4c1e-8778-3022b17478cc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.809535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf6984700f84c04a86fe0d7e4082c99', 'request_id': 'req-3b732afe-7c48-4c1e-8778-3022b17478cc'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.820234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] The token cache is being invalidate because project 4bf6984700f84c04a86fe0d7e4082c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:57:43.820548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.821204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:43.822762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1324/2630] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:57:43 2022] DELETE /identity/v3/projects/4bf6984700f84c04a86fe0d7e4082c99 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:44.031835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:44.033478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:44.045997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec7414-794a-42b1-a63d-608fa04d0a5e service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:44.100670 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:44.101380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:44.101632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:44.101904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:44.103778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:44.167176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:44.168653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:44.178551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1307/2631] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:47.404839 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:47.406425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:47.420214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732afe-7c48-4c1e-8778-3022b17478cc admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:47.424799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:47.425786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:47.426069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:47.426347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] PATH_INFO: `/v3/users/2f3c67a263b14158a01fd906a31696cd` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:47.427567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f3c67a263b14158a01fd906a31696cd)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:47.429559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:47.435626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:47.482210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f3c67a263b14158a01fd906a31696cd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.490642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f3c67a263b14158a01fd906a31696cd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.497497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f3c67a263b14158a01fd906a31696cd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.498120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f3c67a263b14158a01fd906a31696cd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.498715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d14d6f92-75b5-49cf-9c75-8a08912acf4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f3c67a263b14158a01fd906a31696cd'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.515406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1325/2632] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:57:47 2022] DELETE /identity/v3/users/2f3c67a263b14158a01fd906a31696cd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:47.698784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:47.709217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d43a3b-8663-4d2f-845e-6b0eb256cf41 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:47.774093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:47.774959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:47.775236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:47.775506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] PATH_INFO: `/v3/projects/9874821533c3446c9d71a65fb5b8e703` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:47.776741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9874821533c3446c9d71a65fb5b8e703)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:47.778671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:47.806430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:47.833607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9874821533c3446c9d71a65fb5b8e703', 'request_id': 'req-a1374ffd-daae-4a3a-bc4b-69619b7e9925'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.842468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9874821533c3446c9d71a65fb5b8e703', 'request_id': 'req-a1374ffd-daae-4a3a-bc4b-69619b7e9925'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.849090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9874821533c3446c9d71a65fb5b8e703', 'request_id': 'req-a1374ffd-daae-4a3a-bc4b-69619b7e9925'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.859336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] The token cache is being invalidate because project 9874821533c3446c9d71a65fb5b8e703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:57:47.859626 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.860294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-a1374ffd-daae-4a3a-bc4b-69619b7e9925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:57:47.861942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1308/2633] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:57:47 2022] DELETE /identity/v3/projects/9874821533c3446c9d71a65fb5b8e703 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:47.867693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc23d6-65dd-469c-92b9-5a941e2976e6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:47.867922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc23d6-65dd-469c-92b9-5a941e2976e6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:47.868241 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc23d6-65dd-469c-92b9-5a941e2976e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:47.887842 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-97dc23d6-65dd-469c-92b9-5a941e2976e6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:47.889073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-97dc23d6-65dd-469c-92b9-5a941e2976e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:47.952440 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1326/2634] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:47.960932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcdf11-526f-45c1-81aa-170ad0774712 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:47.961215 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcdf11-526f-45c1-81aa-170ad0774712 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:47.961511 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcdf11-526f-45c1-81aa-170ad0774712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:47.980729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-91bcdf11-526f-45c1-81aa-170ad0774712 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:47.981974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-91bcdf11-526f-45c1-81aa-170ad0774712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:47.990310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1309/2635] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:47.994801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc23d6-65dd-469c-92b9-5a941e2976e6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.005031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc23d6-65dd-469c-92b9-5a941e2976e6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.009977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.010624 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.010897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.011164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.011908 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:57:48.012879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.014476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.026617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1327/2636] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:57:48.031025 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcdf11-526f-45c1-81aa-170ad0774712 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.040622 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcdf11-526f-45c1-81aa-170ad0774712 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.045915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.046566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.046846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.047112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.047810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.049763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.080829 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1310/2637] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:57:48 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:57:48.086009 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.102388 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb393acc-b8f5-4cc9-9302-8ef62265ba79 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.168410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.169686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.170184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.170659 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.171806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.174854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.201830 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1328/2638] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:57:48 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:57:48.207713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.218630 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b984a-2e6b-4d83-9c63-228bd02365a3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.223934 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.224645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.224927 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.225204 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.225935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.227540 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.238922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1311/2639] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:48.243154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.248668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:48.253900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5037223-8128-4981-aea7-8109a10c2776 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.259737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ebc1-9f12-4706-a938-17f590cd68ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.260813 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ebc1-9f12-4706-a938-17f590cd68ea admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.261095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ebc1-9f12-4706-a938-17f590cd68ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.261462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ebc1-9f12-4706-a938-17f590cd68ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.262157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ebc1-9f12-4706-a938-17f590cd68ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.264291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ebc1-9f12-4706-a938-17f590cd68ea admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.275329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1329/2640] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:48.279959 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.291650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cff81b-5c6a-42d9-af58-7631ae98c918 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.297190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.298096 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.298377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.298654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] PATH_INFO: `/v3/projects/08d4b87305ee48eba0005eaabf8c50ac/users/01193ff5457b426681f5bbf30eb0f88a/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.314207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d4b87305ee48eba0005eaabf8c50ac, user_id=01193ff5457b426681f5bbf30eb0f88a, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.317030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.334515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1312/2641] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:57:48 2022] PUT /identity/v3/projects/08d4b87305ee48eba0005eaabf8c50ac/users/01193ff5457b426681f5bbf30eb0f88a/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:48.345174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e604b-1472-48ea-9b5a-1c1cb1a925a6 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.345402 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e604b-1472-48ea-9b5a-1c1cb1a925a6 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.345680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e604b-1472-48ea-9b5a-1c1cb1a925a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.372925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-219e604b-1472-48ea-9b5a-1c1cb1a925a6 None None] MFA Rules not processed for user `01193ff5457b426681f5bbf30eb0f88a`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:48.374140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-219e604b-1472-48ea-9b5a-1c1cb1a925a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:48.440413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1330/2642] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:48.444982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.515693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad7b36-b571-4ec3-8c12-5ae8915ba72d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.522488 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.524297 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.524585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.524857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.525558 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.528035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.558219 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1313/2643] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:57:48 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:57:48.562114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e604b-1472-48ea-9b5a-1c1cb1a925a6 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.573702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e604b-1472-48ea-9b5a-1c1cb1a925a6 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.640667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.641278 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.641551 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.641820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.642504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.644542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.667630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1331/2644] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:57:48 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:57:48.671354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.682350 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0a347-8dd2-4f79-9631-e687901deeda admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.687695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.688407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.688686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.688970 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.689742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.691514 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.702520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1314/2645] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:48.706668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.717338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075f2f2-8024-4bf7-850b-55a9bd982d66 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.723751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.724685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.724965 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.725236 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] PATH_INFO: `/v3/projects/5936726bf1704c4eb908bdb849cb4bc3/users/5928971496ee4d0b9e3db3eb5589d206/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.740626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5936726bf1704c4eb908bdb849cb4bc3, user_id=5928971496ee4d0b9e3db3eb5589d206, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.743287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.758831 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1332/2646] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:57:48 2022] PUT /identity/v3/projects/5936726bf1704c4eb908bdb849cb4bc3/users/5928971496ee4d0b9e3db3eb5589d206/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:48.762932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.773520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca500b7b-0880-4a56-9a03-be34669b1fcb admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.787108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:48.836766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3f826-a76e-490c-a5a5-0ad96902231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.837465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f826-a76e-490c-a5a5-0ad96902231f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.837751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f826-a76e-490c-a5a5-0ad96902231f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.838028 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3f826-a76e-490c-a5a5-0ad96902231f admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.838812 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f826-a76e-490c-a5a5-0ad96902231f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.840509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3f826-a76e-490c-a5a5-0ad96902231f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.851258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1315/2647] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:57:48.855763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:48.866557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a4629-2d8d-40ad-9ae2-0c83eba31941 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:48.871673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:48.872574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.872852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.873130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] PATH_INFO: `/v3/projects/5936726bf1704c4eb908bdb849cb4bc3/users/5928971496ee4d0b9e3db3eb5589d206/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.875204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5936726bf1704c4eb908bdb849cb4bc3, user_id=5928971496ee4d0b9e3db3eb5589d206, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:48.877907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:48.888355 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1333/2648] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:57:48 2022] PUT /identity/v3/projects/5936726bf1704c4eb908bdb849cb4bc3/users/5928971496ee4d0b9e3db3eb5589d206/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:57:48.900654 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb335-c3b8-40eb-924d-960745c37bf2 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:48.900846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb335-c3b8-40eb-924d-960745c37bf2 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:48.901255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb335-c3b8-40eb-924d-960745c37bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:48.934804 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-278fb335-c3b8-40eb-924d-960745c37bf2 None None] MFA Rules not processed for user `5928971496ee4d0b9e3db3eb5589d206`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:57:48.936036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-278fb335-c3b8-40eb-924d-960745c37bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:49.008737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1316/2649] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:57:48 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:57:49.022433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:49.024649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:57:49.060904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:49.087029 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c29bd-6645-4b8b-bc6e-87aa6608fdd3 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:49.091253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:49.091904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:49.092219 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:49.092491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:49.103081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:49.104592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:49.114553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1334/2650] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:49.191193 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb335-c3b8-40eb-924d-960745c37bf2 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:57:49.201714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb335-c3b8-40eb-924d-960745c37bf2 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:57:49.205779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:57:49.206430 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:57:49.206716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:57:49.207010 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:57:49.268149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:57:49.269604 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:57:49.282039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1317/2651] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:57:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:57:59.110209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=11 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:57:59.276044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=83 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:21.187467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:21.188922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:21.251207 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4ec27-1319-4a82-802e-cfe5f3b3de0a service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:21.257061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:21.257994 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:21.258350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:21.258640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:21.260751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:21.326787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:21.328275 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:21.340370 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1335/2652] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:58:24.599089 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:24.600649 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:24.614729 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba8693-484b-493f-838d-0d3cc63b3cd5 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:24.619895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:24.620899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:24.621183 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:24.621441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] PATH_INFO: `/v3/users/7f2f23e763704ce2b0969e24ffde9243` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:24.622637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2f23e763704ce2b0969e24ffde9243)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:24.624589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:24.677051 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2f23e763704ce2b0969e24ffde9243'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:24.677705 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2f23e763704ce2b0969e24ffde9243'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:24.689329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2f23e763704ce2b0969e24ffde9243'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:24.698369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2f23e763704ce2b0969e24ffde9243'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:24.699062 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-83d894ce-cfea-428a-86de-c9c407cbf42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2f23e763704ce2b0969e24ffde9243'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:24.712136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1318/2653] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:24 2022] DELETE /identity/v3/users/7f2f23e763704ce2b0969e24ffde9243 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:24.889294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:24.899617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd29e7-b52e-4a51-a5cd-9061397e9dd6 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:24.956024 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:24.956883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:24.957160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:24.957434 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] PATH_INFO: `/v3/projects/f88ea20d31df42ec9e28d8f283aa15ec` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:24.958729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88ea20d31df42ec9e28d8f283aa15ec)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:24.960686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.011727 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88ea20d31df42ec9e28d8f283aa15ec', 'request_id': 'req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:25.014085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:25.018451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88ea20d31df42ec9e28d8f283aa15ec', 'request_id': 'req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:25.033799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88ea20d31df42ec9e28d8f283aa15ec', 'request_id': 'req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:25.045387 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] The token cache is being invalidate because project f88ea20d31df42ec9e28d8f283aa15ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:25.045670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:25.046563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1bd079d4-5ba3-4633-bc6a-7e208a2ee466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:25.048362 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1336/2654] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:24 2022] DELETE /identity/v3/projects/f88ea20d31df42ec9e28d8f283aa15ec => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:25.053498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c0a08-d33e-4a0d-9eb9-941d1dd0b6fc None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.053789 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c0a08-d33e-4a0d-9eb9-941d1dd0b6fc None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.054094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c0a08-d33e-4a0d-9eb9-941d1dd0b6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.074328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0f9c0a08-d33e-4a0d-9eb9-941d1dd0b6fc None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:25.075620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0f9c0a08-d33e-4a0d-9eb9-941d1dd0b6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:25.143865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1319/2655] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:25.152630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8077-6ae1-492f-92ac-14d6508b5511 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.152914 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8077-6ae1-492f-92ac-14d6508b5511 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.153193 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8077-6ae1-492f-92ac-14d6508b5511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.173694 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-6a7e8077-6ae1-492f-92ac-14d6508b5511 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:25.174827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-6a7e8077-6ae1-492f-92ac-14d6508b5511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:25.183525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1337/2656] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:25.188109 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c0a08-d33e-4a0d-9eb9-941d1dd0b6fc None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:25.197810 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c0a08-d33e-4a0d-9eb9-941d1dd0b6fc None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:25.202609 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:25.203348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.203687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.204045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.204839 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:58:25.205202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:25.206863 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.219194 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1320/2657] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:58:25.224124 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e8077-6ae1-492f-92ac-14d6508b5511 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:25.233894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e8077-6ae1-492f-92ac-14d6508b5511 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:25.238315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:25.238947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.239225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.239503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.240327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:25.242254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.270026 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1338/2658] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:25 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:25.273908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:25.290127 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9740ce6-3d60-443d-9ac3-8e19959d2b2e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:25.350559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:25.351336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.351653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.352038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.352742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:25.354773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.361619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:25.378872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1321/2659] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:25 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:25.383212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:25.393925 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917b9b4-3d4b-4852-838e-65263c203d31 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:25.398679 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:25.399340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.399623 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.399897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.400857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:25.402530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.413004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1339/2660] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:25.418855 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:25.429639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebb0cb0-f1c5-4f41-a72c-034b3edc2c40 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:25.435632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e5162-a929-4447-944a-ee87635f8a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:25.436349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5162-a929-4447-944a-ee87635f8a35 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.436624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5162-a929-4447-944a-ee87635f8a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.436990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e5162-a929-4447-944a-ee87635f8a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.437735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5162-a929-4447-944a-ee87635f8a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:25.439345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e5162-a929-4447-944a-ee87635f8a35 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.450064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1322/2661] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:25.454446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:25.465495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ac3e5-14ee-43df-b46d-f085d19f1827 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:25.471455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:25.472714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.472991 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.473270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] PATH_INFO: `/v3/projects/6c05e9d2132f446a93aba1f27c37b412/users/9ff354a709914b99bee9785b9577781d/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.489843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c05e9d2132f446a93aba1f27c37b412, user_id=9ff354a709914b99bee9785b9577781d, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:25.492548 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.508808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1340/2662] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:25 2022] PUT /identity/v3/projects/6c05e9d2132f446a93aba1f27c37b412/users/9ff354a709914b99bee9785b9577781d/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:25.514986 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.515267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.515631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.538255 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:25.539476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:25.613191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1323/2663] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:25.623397 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:25.625395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:25.683577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31545e12-d429-4b31-b619-4860656e554a admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:25.687658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a30de-1bbd-4525-a469-58c74d8f2ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:25.688354 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a30de-1bbd-4525-a469-58c74d8f2ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:25.688627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a30de-1bbd-4525-a469-58c74d8f2ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:25.688899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a30de-1bbd-4525-a469-58c74d8f2ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:25.698382 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a30de-1bbd-4525-a469-58c74d8f2ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:25.699819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a30de-1bbd-4525-a469-58c74d8f2ae4 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:25.709525 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1341/2664] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:28.125326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:28.126788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:28.140553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1158a58c-2106-463e-84ec-aff3ce317367 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:28.145237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:28.146131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:28.146410 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:28.146681 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] PATH_INFO: `/v3/users/01193ff5457b426681f5bbf30eb0f88a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:28.147871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01193ff5457b426681f5bbf30eb0f88a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:28.150016 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:28.205312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01193ff5457b426681f5bbf30eb0f88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:28.206407 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01193ff5457b426681f5bbf30eb0f88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:28.221357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01193ff5457b426681f5bbf30eb0f88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:28.232004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01193ff5457b426681f5bbf30eb0f88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:28.233175 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01193ff5457b426681f5bbf30eb0f88a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:28.254299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1324/2665] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:28 2022] DELETE /identity/v3/users/01193ff5457b426681f5bbf30eb0f88a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:28.259764 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:28.261390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:28.261929 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:28.286711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:28.288256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:28.356249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1342/2666] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:28.368632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:28.370583 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:28.481951 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f525e-5c38-48a5-b5a0-535cbd1f2cb0 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:28.486480 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:28.487299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:28.487587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:28.487861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:28.498094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:28.504330 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:28.506607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:28.550943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1325/2667] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:28.975145 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:28.976902 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:28.990010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ca24b-4956-4d48-be72-d564777dbf69 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:28.996289 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:28.997295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:28.997707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:28.998080 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] PATH_INFO: `/v3/projects/08d4b87305ee48eba0005eaabf8c50ac` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:28.999417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d4b87305ee48eba0005eaabf8c50ac)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:29.001651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:29.021016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:29.071485 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d4b87305ee48eba0005eaabf8c50ac', 'request_id': 'req-b4883b5b-7889-4a0b-90da-bd14432a941d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.095146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d4b87305ee48eba0005eaabf8c50ac', 'request_id': 'req-b4883b5b-7889-4a0b-90da-bd14432a941d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.112741 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d4b87305ee48eba0005eaabf8c50ac', 'request_id': 'req-b4883b5b-7889-4a0b-90da-bd14432a941d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.138155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] The token cache is being invalidate because project 08d4b87305ee48eba0005eaabf8c50ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:29.138447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.139133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.141056 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1343/2668] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:28 2022] DELETE /identity/v3/projects/08d4b87305ee48eba0005eaabf8c50ac => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:29.145250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:29.164064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850301f2-032b-480a-98f2-fb919ea022f5 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:29.243850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:29.244928 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:29.245258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:29.245606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] PATH_INFO: `/v3/users/5928971496ee4d0b9e3db3eb5589d206` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:29.248929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5928971496ee4d0b9e3db3eb5589d206)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:29.251206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:29.327213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5928971496ee4d0b9e3db3eb5589d206'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.327946 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5928971496ee4d0b9e3db3eb5589d206'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.348464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5928971496ee4d0b9e3db3eb5589d206'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.361737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5928971496ee4d0b9e3db3eb5589d206'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.362536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-3b08610f-fa4d-42fb-98dc-1a0bcc8e21ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5928971496ee4d0b9e3db3eb5589d206'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.381342 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1326/2669] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:29 2022] DELETE /identity/v3/users/5928971496ee4d0b9e3db3eb5589d206 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:29.696755 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:29.698527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:29.716691 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4883b5b-7889-4a0b-90da-bd14432a941d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:29.805073 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:29.805979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:29.807926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:29.808399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] PATH_INFO: `/v3/projects/5936726bf1704c4eb908bdb849cb4bc3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:29.810320 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5936726bf1704c4eb908bdb849cb4bc3)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:29.812706 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:29.903344 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5936726bf1704c4eb908bdb849cb4bc3', 'request_id': 'req-a3503a63-37b2-4090-a314-f5d89400aedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.911973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5936726bf1704c4eb908bdb849cb4bc3', 'request_id': 'req-a3503a63-37b2-4090-a314-f5d89400aedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.924523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5936726bf1704c4eb908bdb849cb4bc3', 'request_id': 'req-a3503a63-37b2-4090-a314-f5d89400aedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.945943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] The token cache is being invalidate because project 5936726bf1704c4eb908bdb849cb4bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:29.945943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.945943 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-a3503a63-37b2-4090-a314-f5d89400aedf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:29.946522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1344/2670] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:29 2022] DELETE /identity/v3/projects/5936726bf1704c4eb908bdb849cb4bc3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:29.950957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f81cc1-9634-4b38-bb42-d2f6e066ba03 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:29.951235 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:29.951345 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f81cc1-9634-4b38-bb42-d2f6e066ba03 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:29.951548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:29.951743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f81cc1-9634-4b38-bb42-d2f6e066ba03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:29.951858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:29.970754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-06f81cc1-9634-4b38-bb42-d2f6e066ba03 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:29.972592 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-06f81cc1-9634-4b38-bb42-d2f6e066ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:29.984182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] MFA Rules not processed for user `9ff354a709914b99bee9785b9577781d`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:29.985628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:30.010996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:30.058213 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1345/2671] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:30.066187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192f24-cc26-420a-9fb4-9b76fddbd2f7 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.066474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192f24-cc26-420a-9fb4-9b76fddbd2f7 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.066876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7192f24-cc26-420a-9fb4-9b76fddbd2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.076124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1327/2672] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:30.087439 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-d7192f24-cc26-420a-9fb4-9b76fddbd2f7 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:30.088894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d7192f24-cc26-420a-9fb4-9b76fddbd2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:30.091903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.093428 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:30.098660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1346/2673] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:30.099310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:30.102781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192f24-cc26-420a-9fb4-9b76fddbd2f7 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.112964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7192f24-cc26-420a-9fb4-9b76fddbd2f7 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.118576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.119361 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.119641 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.120033 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.120765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:58:30.121071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.122674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.138290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1347/2674] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:58:30.139648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.150435 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47ab8b-6440-4f09-bbe1-7233d958ca93 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.155224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.155879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.156315 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.156595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.157294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.159311 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.175887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c48b2-85da-4f36-991a-60ef17d5cba8 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.188385 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.194228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.194832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.195287 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.197018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1348/2675] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:30 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:58:30.201183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.216895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.218924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.230031 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1328/2676] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:30.294208 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1cce7-4f2c-4161-8bf4-e5093c6e39ab admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.300738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.301633 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.302070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.302536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.303407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.305977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.331893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1349/2677] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:30 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:58:30.335846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.347944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f603c-3ca5-48cd-84db-58a34af74738 service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.353049 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.353773 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.354052 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.354384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.355123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.357073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.371363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1329/2678] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:58:30.372760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.385825 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1480e7-e082-45de-bbc1-9411ea46fa7e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.391185 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa0b0f-efa5-4505-8f7c-bc9e690295d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.391978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa0b0f-efa5-4505-8f7c-bc9e690295d8 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.391978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa0b0f-efa5-4505-8f7c-bc9e690295d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.392211 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa0b0f-efa5-4505-8f7c-bc9e690295d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.392934 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa0b0f-efa5-4505-8f7c-bc9e690295d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.394555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa0b0f-efa5-4505-8f7c-bc9e690295d8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.414556 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1350/2679] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:30.418534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.435265 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268c578-7d42-4194-a696-2beb3114076a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.440565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.441571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.441797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.442072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] PATH_INFO: `/v3/projects/43b9a787a1a14ba48109a7fc15747dc8/users/fbcb60e04b9b4a22bd95cc804f8f66a3/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.463180 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b9a787a1a14ba48109a7fc15747dc8, user_id=fbcb60e04b9b4a22bd95cc804f8f66a3, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.478351 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.504509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1330/2680] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:30 2022] PUT /identity/v3/projects/43b9a787a1a14ba48109a7fc15747dc8/users/fbcb60e04b9b4a22bd95cc804f8f66a3/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:30.516170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c6f0-254f-4a15-98f8-d6a0c92cf7d5 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.516464 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c6f0-254f-4a15-98f8-d6a0c92cf7d5 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.516743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0c6f0-254f-4a15-98f8-d6a0c92cf7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.549619 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-dea0c6f0-254f-4a15-98f8-d6a0c92cf7d5 None None] MFA Rules not processed for user `fbcb60e04b9b4a22bd95cc804f8f66a3`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:30.550968 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-dea0c6f0-254f-4a15-98f8-d6a0c92cf7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:30.630010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:30.636931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1351/2681] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:30.642018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.728013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c5b2e-634a-4e4b-9a7b-ee4248989db8 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.732880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.734138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.734138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.734138 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.734914 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.736872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.769304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1331/2682] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:30 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:30.773483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c6f0-254f-4a15-98f8-d6a0c92cf7d5 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.802383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0c6f0-254f-4a15-98f8-d6a0c92cf7d5 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.880805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.881503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.881782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.882060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.882799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.885034 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.909307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1352/2683] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:30 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:30.913158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.925807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba3ffe-343e-435c-a73c-a8084bc1356d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.932509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.933196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.933469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.933741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.934483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.936131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:30.948355 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1332/2684] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:58:30.952715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:30.964174 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459c7721-846e-4cef-bf80-c671b0fdfdc0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:30.969168 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:30.970225 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:30.970510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:30.970808 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] PATH_INFO: `/v3/projects/a8fb547ab1fd4904865f2ce85423ada1/users/0736c3b05bb841a38ee8dfb32e28691b/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:30.988868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8fb547ab1fd4904865f2ce85423ada1, user_id=0736c3b05bb841a38ee8dfb32e28691b, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:30.991561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:31.007785 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1353/2685] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:30 2022] PUT /identity/v3/projects/a8fb547ab1fd4904865f2ce85423ada1/users/0736c3b05bb841a38ee8dfb32e28691b/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:58:31.012032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:31.022440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1c2c9-8b9f-4c79-949c-5374fa82c1fc admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:31.050038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:31.090534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988bde6-6c16-4624-a137-7fb2fc375e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:31.091334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988bde6-6c16-4624-a137-7fb2fc375e04 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:31.091686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988bde6-6c16-4624-a137-7fb2fc375e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:31.092704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988bde6-6c16-4624-a137-7fb2fc375e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:31.093131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988bde6-6c16-4624-a137-7fb2fc375e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:31.095313 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988bde6-6c16-4624-a137-7fb2fc375e04 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:31.106388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1333/2686] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:31.110873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:31.121651 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ede290-13e7-40c3-b62f-4cbe611d5412 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:31.126531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:31.127559 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:31.127930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:31.128407 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] PATH_INFO: `/v3/projects/a8fb547ab1fd4904865f2ce85423ada1/users/0736c3b05bb841a38ee8dfb32e28691b/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:31.130658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8fb547ab1fd4904865f2ce85423ada1, user_id=0736c3b05bb841a38ee8dfb32e28691b, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:31.133340 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:31.144927 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1354/2687] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:31 2022] PUT /identity/v3/projects/a8fb547ab1fd4904865f2ce85423ada1/users/0736c3b05bb841a38ee8dfb32e28691b/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:31.155739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a0db4-2090-45bf-ba81-a316d4fd8155 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:31.158489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a0db4-2090-45bf-ba81-a316d4fd8155 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:31.158875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a0db4-2090-45bf-ba81-a316d4fd8155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:31.197343 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-3e7a0db4-2090-45bf-ba81-a316d4fd8155 None None] MFA Rules not processed for user `0736c3b05bb841a38ee8dfb32e28691b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:31.198650 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-3e7a0db4-2090-45bf-ba81-a316d4fd8155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:31.292770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1334/2688] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:31.304396 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:31.306327 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:31.374580 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfb3f9-f3ac-48a1-8aa7-367ff5b4bb16 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:31.378854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca007a7-6963-4c31-bf32-c247fdabcd43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:31.379535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca007a7-6963-4c31-bf32-c247fdabcd43 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:31.379810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca007a7-6963-4c31-bf32-c247fdabcd43 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:31.380129 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca007a7-6963-4c31-bf32-c247fdabcd43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:31.395433 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca007a7-6963-4c31-bf32-c247fdabcd43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:31.397063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca007a7-6963-4c31-bf32-c247fdabcd43 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:31.407713 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1355/2689] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:31.498857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a0db4-2090-45bf-ba81-a316d4fd8155 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:31.509157 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a0db4-2090-45bf-ba81-a316d4fd8155 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:31.512977 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:31.513728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:31.514077 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:31.514417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:31.574238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:31.575766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:31.585358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1335/2690] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:31.696877 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9672213-38b9-4184-9dba-7c77ed5f587b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:31.697175 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9672213-38b9-4184-9dba-7c77ed5f587b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:31.697430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9672213-38b9-4184-9dba-7c77ed5f587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:31.717876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a9672213-38b9-4184-9dba-7c77ed5f587b None None] MFA Rules not processed for user `9ff354a709914b99bee9785b9577781d`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:31.719136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a9672213-38b9-4184-9dba-7c77ed5f587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:31.774513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:31.784141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1356/2691] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:40.212135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:40.222148 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab9feb1-5af4-40dc-a9b9-f26c71ac9dd9 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:40.226899 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570f545-5e00-4737-ab21-190616f77f91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:40.227546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f545-5e00-4737-ab21-190616f77f91 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:40.227833 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f545-5e00-4737-ab21-190616f77f91 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:40.228158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570f545-5e00-4737-ab21-190616f77f91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:40.230346 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0570f545-5e00-4737-ab21-190616f77f91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:40.278912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:40.294644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f545-5e00-4737-ab21-190616f77f91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:40.296168 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570f545-5e00-4737-ab21-190616f77f91 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:40.305731 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1336/2692] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:40.593605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9672213-38b9-4184-9dba-7c77ed5f587b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:40.658057 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9672213-38b9-4184-9dba-7c77ed5f587b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:40.663497 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:40.664503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:40.664791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:40.665046 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] PATH_INFO: `/v3/users/fbcb60e04b9b4a22bd95cc804f8f66a3` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:40.666368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcb60e04b9b4a22bd95cc804f8f66a3)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:40.668731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:40.722004 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcb60e04b9b4a22bd95cc804f8f66a3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:40.732491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcb60e04b9b4a22bd95cc804f8f66a3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:40.741076 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcb60e04b9b4a22bd95cc804f8f66a3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:40.741792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcb60e04b9b4a22bd95cc804f8f66a3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:40.742425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcb60e04b9b4a22bd95cc804f8f66a3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:40.760186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1357/2693] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:40 2022] DELETE /identity/v3/users/fbcb60e04b9b4a22bd95cc804f8f66a3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:40.765166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:40.765450 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:40.765730 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:40.785947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:40.787567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:40.849799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1337/2694] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:40.859848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:40.861418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:40.919765 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c6d09-34a3-448f-8cde-b1176395e572 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:40.923797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:40.924432 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:40.924711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:40.924978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:40.934250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:40.935770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:40.945115 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1358/2695] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:41.234069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:41.235766 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:41.248369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a6ece-768d-4616-b76e-ffddb98392cb None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:41.254218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:41.255206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:41.255486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:41.255740 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] PATH_INFO: `/v3/projects/43b9a787a1a14ba48109a7fc15747dc8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:41.257058 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b9a787a1a14ba48109a7fc15747dc8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:41.258965 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:41.308874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b9a787a1a14ba48109a7fc15747dc8', 'request_id': 'req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.317794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b9a787a1a14ba48109a7fc15747dc8', 'request_id': 'req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.324548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b9a787a1a14ba48109a7fc15747dc8', 'request_id': 'req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.335034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] The token cache is being invalidate because project 43b9a787a1a14ba48109a7fc15747dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:41.335316 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.335972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.337592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1338/2696] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:41 2022] DELETE /identity/v3/projects/43b9a787a1a14ba48109a7fc15747dc8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:41.341886 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:41.352338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd863b-5481-4d37-a276-e34ac8cd8b5e service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:41.371555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:41.413183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:41.414063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:41.414341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:41.414615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] PATH_INFO: `/v3/users/0736c3b05bb841a38ee8dfb32e28691b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:41.415841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0736c3b05bb841a38ee8dfb32e28691b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:41.417862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:41.467099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0736c3b05bb841a38ee8dfb32e28691b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.476383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0736c3b05bb841a38ee8dfb32e28691b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.484317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0736c3b05bb841a38ee8dfb32e28691b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.485019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0736c3b05bb841a38ee8dfb32e28691b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.485750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-667257c6-a17f-47ff-9f27-b71d6a52daaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0736c3b05bb841a38ee8dfb32e28691b'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.506406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1359/2697] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:41 2022] DELETE /identity/v3/users/0736c3b05bb841a38ee8dfb32e28691b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:41.798574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:41.800095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:41.814613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d66ec-1e39-4d96-a71d-45d98e2cb42c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:41.879080 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:41.880293 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:41.880435 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:41.880721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] PATH_INFO: `/v3/projects/a8fb547ab1fd4904865f2ce85423ada1` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:41.881932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8fb547ab1fd4904865f2ce85423ada1)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:41.883920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:41.895573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:41.938690 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8fb547ab1fd4904865f2ce85423ada1', 'request_id': 'req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.947798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8fb547ab1fd4904865f2ce85423ada1', 'request_id': 'req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.954401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8fb547ab1fd4904865f2ce85423ada1', 'request_id': 'req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.964814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] The token cache is being invalidate because project a8fb547ab1fd4904865f2ce85423ada1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:41.965102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.965742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-bcf9ed23-8989-4d93-8bbb-ac7172438ebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:41.967317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1339/2698] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:41 2022] DELETE /identity/v3/projects/a8fb547ab1fd4904865f2ce85423ada1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:41.973599 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd4a28-24d0-4d89-939f-027e8f776412 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:41.974122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd4a28-24d0-4d89-939f-027e8f776412 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:41.974122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd4a28-24d0-4d89-939f-027e8f776412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:41.993209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-12fd4a28-24d0-4d89-939f-027e8f776412 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:41.993998 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-12fd4a28-24d0-4d89-939f-027e8f776412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:42.058736 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1360/2699] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:42.067079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b010-91c1-4756-95b5-4faad153c4b5 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.067471 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b010-91c1-4756-95b5-4faad153c4b5 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.067821 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336b010-91c1-4756-95b5-4faad153c4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.086699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-d336b010-91c1-4756-95b5-4faad153c4b5 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:42.087921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d336b010-91c1-4756-95b5-4faad153c4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:42.097737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1340/2700] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:42.102104 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd4a28-24d0-4d89-939f-027e8f776412 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.112038 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd4a28-24d0-4d89-939f-027e8f776412 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.116667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.117321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.117605 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.117880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.118610 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:58:42.118918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.120593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.132303 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1361/2701] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:58:42.136602 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b010-91c1-4756-95b5-4faad153c4b5 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.146214 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336b010-91c1-4756-95b5-4faad153c4b5 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.152607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.153261 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.153546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.153865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.154567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.156515 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.189044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1341/2702] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:42 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 15:58:42.190171 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.202202 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbe4cf-afa6-49b2-9172-5eac916a8c31 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.228466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:42.264751 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.265348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.265627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.265926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.266635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.268598 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.290682 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1362/2703] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:42 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:42.295065 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.305754 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea1709-65a4-45c7-add2-431867572b58 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.310886 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.311545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.311838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.312123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.312845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.314437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.327012 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1342/2704] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:42.331926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.344973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787649ae-e0fb-4cef-936d-c2a8365d9c89 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.350587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf45d8-5a40-4dd5-98ff-38af16ea92be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.351246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf45d8-5a40-4dd5-98ff-38af16ea92be admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.351528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf45d8-5a40-4dd5-98ff-38af16ea92be admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.351801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf45d8-5a40-4dd5-98ff-38af16ea92be admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.352589 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf45d8-5a40-4dd5-98ff-38af16ea92be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.354210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf45d8-5a40-4dd5-98ff-38af16ea92be admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.364430 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1363/2705] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:42.369000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.379727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a2334-fc3c-40c8-9c69-7cdf75989a8f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.385778 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.386674 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.386956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.387229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] PATH_INFO: `/v3/projects/4200d2d90a3e476aba5bb719305ef162/users/45e7dce40a9e49bcb769461f1f62d66a/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.402949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4200d2d90a3e476aba5bb719305ef162, user_id=45e7dce40a9e49bcb769461f1f62d66a, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.405635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.420272 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1343/2706] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:42 2022] PUT /identity/v3/projects/4200d2d90a3e476aba5bb719305ef162/users/45e7dce40a9e49bcb769461f1f62d66a/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:42.431735 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5a4ca-5a09-4c45-90bc-e98b252bee65 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.432030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5a4ca-5a09-4c45-90bc-e98b252bee65 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.432338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5a4ca-5a09-4c45-90bc-e98b252bee65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.459707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7ad5a4ca-5a09-4c45-90bc-e98b252bee65 None None] MFA Rules not processed for user `45e7dce40a9e49bcb769461f1f62d66a`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:42.461006 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7ad5a4ca-5a09-4c45-90bc-e98b252bee65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:42.533022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1364/2707] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:42.534363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.599599 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fcf7b-9628-40e9-a669-8422a0cb0f7a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.604463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.605075 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.605356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.605627 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.606320 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.608241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.639391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1344/2708] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:42 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:42.643571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5a4ca-5a09-4c45-90bc-e98b252bee65 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.654542 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5a4ca-5a09-4c45-90bc-e98b252bee65 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.712415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.713088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.713365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.713635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.714338 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.716348 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.739065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1365/2709] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:42 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:42.743964 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.757246 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7ec48-8e12-4f36-8ffe-be91fa2103b8 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.762259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.762929 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.763213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.763485 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.764250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.766102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.778251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1345/2710] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:58:42.784212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.795114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14810389-4917-4527-842e-9d571ddac7e6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.801001 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.801892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.802152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.802423 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] PATH_INFO: `/v3/projects/a4732a08772e4c70ac747c547320d51c/users/27367c6c8f634ad1bf4c27f49d4767b8/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.818346 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4732a08772e4c70ac747c547320d51c, user_id=27367c6c8f634ad1bf4c27f49d4767b8, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.821103 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.838085 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1366/2711] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:42 2022] PUT /identity/v3/projects/a4732a08772e4c70ac747c547320d51c/users/27367c6c8f634ad1bf4c27f49d4767b8/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:42.842249 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.853675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22c5ea-7e41-4f3c-9ae4-d1989264687f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.858727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:42.913003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797fe1c-4063-4ac5-a10a-0e5bf73b22e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.913692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797fe1c-4063-4ac5-a10a-0e5bf73b22e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.913969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797fe1c-4063-4ac5-a10a-0e5bf73b22e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.914224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797fe1c-4063-4ac5-a10a-0e5bf73b22e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.914940 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797fe1c-4063-4ac5-a10a-0e5bf73b22e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.916568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797fe1c-4063-4ac5-a10a-0e5bf73b22e4 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.926448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1346/2712] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:42.931032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:42.938547 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:42.946676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3f8c2-c290-4a1b-8f53-ac20844ece9d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:42.955086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:42.956394 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.956740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.957146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] PATH_INFO: `/v3/projects/a4732a08772e4c70ac747c547320d51c/users/27367c6c8f634ad1bf4c27f49d4767b8/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:42.960342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4732a08772e4c70ac747c547320d51c, user_id=27367c6c8f634ad1bf4c27f49d4767b8, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:42.964386 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:42.975513 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1367/2713] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:42 2022] PUT /identity/v3/projects/a4732a08772e4c70ac747c547320d51c/users/27367c6c8f634ad1bf4c27f49d4767b8/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:42.986323 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ad5d-575f-481c-a17e-562f6cca2531 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:42.986618 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ad5d-575f-481c-a17e-562f6cca2531 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:42.986915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ad5d-575f-481c-a17e-562f6cca2531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:43.013917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b532ad5d-575f-481c-a17e-562f6cca2531 None None] MFA Rules not processed for user `27367c6c8f634ad1bf4c27f49d4767b8`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:43.015123 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b532ad5d-575f-481c-a17e-562f6cca2531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:43.086203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1347/2714] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:43.098796 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:43.100757 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:43.160730 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1355-a9bf-4cb8-b58d-0b670f2eea46 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:43.164431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:43.165086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:43.165365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:43.165654 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:43.179096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:43.180574 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:43.190597 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1368/2715] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:43.546920 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532ad5d-575f-481c-a17e-562f6cca2531 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:43.570325 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532ad5d-575f-481c-a17e-562f6cca2531 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:43.575799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27367c6c8f634ad1bf4c27f49d4767b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4732a08772e4c70ac747c547320d51c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:43.576787 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:43.577336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:43.577823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:43.578857 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:58:43.579395 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:43.581277 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:43.602763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1348/2716] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:58:43.604672 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:43.615966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57fe53-897d-44cd-9bdc-df08a99b2960 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:43.621013 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27367c6c8f634ad1bf4c27f49d4767b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4732a08772e4c70ac747c547320d51c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:43.621844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:43.622012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:43.622353 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:43.623152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:43.625106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:43.656572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1369/2717] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:43 2022] POST /identity/v3/projects => generated 354 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:43.660648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:43.673070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a10267-bee3-40f5-8425-b99e72ff6f61 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:43.737507 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27367c6c8f634ad1bf4c27f49d4767b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4732a08772e4c70ac747c547320d51c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:43.738726 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:43.739013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:43.739304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:43.740060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:58:43.740388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:43.742000 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:43.757020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1349/2718] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:58:43.792703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:43.804431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12159c-2626-4da5-96e2-8da8ae39cfe9 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:43.811491 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '27367c6c8f634ad1bf4c27f49d4767b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a4732a08772e4c70ac747c547320d51c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:43.812534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:43.812880 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:43.813234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] PATH_INFO: `/v3/projects/ae29b27d71fb40da8d657058acf64032` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:43.814530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ae29b27d71fb40da8d657058acf64032)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:43.816634 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:43.874901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae29b27d71fb40da8d657058acf64032', 'request_id': 'req-f1e35369-7319-407c-bbdc-95bade612980'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:43.885806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae29b27d71fb40da8d657058acf64032', 'request_id': 'req-f1e35369-7319-407c-bbdc-95bade612980'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:43.895849 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae29b27d71fb40da8d657058acf64032', 'request_id': 'req-f1e35369-7319-407c-bbdc-95bade612980'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:43.908456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] The token cache is being invalidate because project ae29b27d71fb40da8d657058acf64032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:43.909016 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:43.909674 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:43.911591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1370/2719] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:43 2022] DELETE /identity/v3/projects/ae29b27d71fb40da8d657058acf64032 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:44.130872 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:44.132770 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:44.146844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:44.204088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9c3a1-f9a1-473e-b9f5-e8e41afc5099 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:44.228737 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:44.230212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:44.265466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:44.266267 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:44.266542 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:44.266815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] PATH_INFO: `/v3/users/45e7dce40a9e49bcb769461f1f62d66a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:44.268029 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e7dce40a9e49bcb769461f1f62d66a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:44.269942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:44.287277 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:44.308846 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e35369-7319-407c-bbdc-95bade612980 tempest-VolumeQuotaClassesTest-296970445 tempest-VolumeQuotaClassesTest-296970445-project-admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:44.315763 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd665c3-808d-47b9-8c3f-64b8b5da55ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:44.316500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd665c3-808d-47b9-8c3f-64b8b5da55ba service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:44.316781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd665c3-808d-47b9-8c3f-64b8b5da55ba service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:44.317074 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd665c3-808d-47b9-8c3f-64b8b5da55ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:44.318973 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-5bd665c3-808d-47b9-8c3f-64b8b5da55ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:44.324451 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e7dce40a9e49bcb769461f1f62d66a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.325768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e7dce40a9e49bcb769461f1f62d66a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.331568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd665c3-808d-47b9-8c3f-64b8b5da55ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:44.333712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd665c3-808d-47b9-8c3f-64b8b5da55ba service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:44.343782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e7dce40a9e49bcb769461f1f62d66a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.345573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1371/2720] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:44.352591 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e7dce40a9e49bcb769461f1f62d66a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.353209 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e7dce40a9e49bcb769461f1f62d66a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.366524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1350/2721] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:44 2022] DELETE /identity/v3/users/45e7dce40a9e49bcb769461f1f62d66a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:44.370844 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:44.371133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:44.371408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:44.389797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:44.392030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:44.456745 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1372/2722] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:44.466092 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:44.467517 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:44.510843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:44.527402 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eccf44-e77d-4183-8288-3fcc9c540e1a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:44.531072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:44.531716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:44.532013 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:44.532295 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:44.541704 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:44.543190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:44.552942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1351/2723] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:44.835450 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:44.836988 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:44.854655 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd4acb-7030-4668-a62b-4837ccd13813 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:44.859958 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:44.860853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:44.861128 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:44.861395 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] PATH_INFO: `/v3/projects/4200d2d90a3e476aba5bb719305ef162` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:44.862591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4200d2d90a3e476aba5bb719305ef162)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:44.864772 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:44.915183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4200d2d90a3e476aba5bb719305ef162', 'request_id': 'req-d73ebdb2-fde3-456f-99f0-d6eb43e52392'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.921479 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4200d2d90a3e476aba5bb719305ef162', 'request_id': 'req-d73ebdb2-fde3-456f-99f0-d6eb43e52392'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.931110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4200d2d90a3e476aba5bb719305ef162', 'request_id': 'req-d73ebdb2-fde3-456f-99f0-d6eb43e52392'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.942017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] The token cache is being invalidate because project 4200d2d90a3e476aba5bb719305ef162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:44.942294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.942921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:44.944522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1373/2724] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:44 2022] DELETE /identity/v3/projects/4200d2d90a3e476aba5bb719305ef162 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:44.948798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:44.958774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb98c395-1700-4ac7-a460-38718edca1e7 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:45.020382 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:45.021275 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.021581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.021858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] PATH_INFO: `/v3/users/27367c6c8f634ad1bf4c27f49d4767b8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.023122 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=27367c6c8f634ad1bf4c27f49d4767b8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:45.025104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:45.083264 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27367c6c8f634ad1bf4c27f49d4767b8'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.084120 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27367c6c8f634ad1bf4c27f49d4767b8'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.097544 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27367c6c8f634ad1bf4c27f49d4767b8'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.106502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27367c6c8f634ad1bf4c27f49d4767b8'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.107135 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-d1398b9c-b67d-4ed8-bee3-88d36bb6dfdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27367c6c8f634ad1bf4c27f49d4767b8'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.119532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1352/2725] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:44 2022] DELETE /identity/v3/users/27367c6c8f634ad1bf4c27f49d4767b8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:45.415788 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:45.417871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:45.436324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ebdb2-fde3-456f-99f0-d6eb43e52392 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:45.516105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:45.517259 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.517539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.517810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] PATH_INFO: `/v3/projects/a4732a08772e4c70ac747c547320d51c` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.519584 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4732a08772e4c70ac747c547320d51c)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:45.521002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:45.531053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:45.575495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4732a08772e4c70ac747c547320d51c', 'request_id': 'req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.581738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4732a08772e4c70ac747c547320d51c', 'request_id': 'req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.592695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4732a08772e4c70ac747c547320d51c', 'request_id': 'req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.604585 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] The token cache is being invalidate because project a4732a08772e4c70ac747c547320d51c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:45.604951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.605587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-f8387ee3-bb7f-40dc-a1b3-6bcb768fe810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:45.607161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1374/2726] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:45 2022] DELETE /identity/v3/projects/a4732a08772e4c70ac747c547320d51c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:45.613035 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ae522-0990-4c91-9f72-aba385ea7c08 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.613573 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ae522-0990-4c91-9f72-aba385ea7c08 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.613675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ae522-0990-4c91-9f72-aba385ea7c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.631298 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-949ae522-0990-4c91-9f72-aba385ea7c08 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:45.632564 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-949ae522-0990-4c91-9f72-aba385ea7c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:45.687056 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:45.705130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1353/2727] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:45.707447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d2ecb-0d10-4f7b-b8ca-039067f12c78 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.707720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d2ecb-0d10-4f7b-b8ca-039067f12c78 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.708018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d2ecb-0d10-4f7b-b8ca-039067f12c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.726680 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-8f2d2ecb-0d10-4f7b-b8ca-039067f12c78 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:45.727888 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-8f2d2ecb-0d10-4f7b-b8ca-039067f12c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:45.736454 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1375/2728] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:45.740865 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ae522-0990-4c91-9f72-aba385ea7c08 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:45.751003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ae522-0990-4c91-9f72-aba385ea7c08 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:45.756491 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:45.757152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.757437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.757708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.758441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:58:45.758748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:45.760388 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:45.772749 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1354/2729] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 15:58:45.777457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d2ecb-0d10-4f7b-b8ca-039067f12c78 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:45.786850 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d2ecb-0d10-4f7b-b8ca-039067f12c78 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:45.792816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:45.793474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.793752 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.794019 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.794710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:45.796710 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:45.828322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1376/2730] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:45 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:45.832767 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:45.843344 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65fa8c-18c5-440b-a9e5-9927e8d66ad5 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:45.904136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:45.904807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.905087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.905361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.906060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:45.908019 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:45.931438 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1355/2731] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:45 2022] POST /identity/v3/users => generated 305 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:45.935697 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:45.946950 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e35c4a2-461d-4b35-9ec0-d05e6c790ab5 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:45.952723 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:45.953130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.953514 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.953793 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.954670 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:45.956606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:45.967900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1377/2732] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:45.972387 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:45.983206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26abff39-b6e9-43e9-bfc0-7726429e7bb6 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:45.989097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d82ae6-f38a-405b-b6dc-764eb627804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:45.989768 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d82ae6-f38a-405b-b6dc-764eb627804a admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:45.990066 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d82ae6-f38a-405b-b6dc-764eb627804a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:45.990334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d82ae6-f38a-405b-b6dc-764eb627804a admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:45.991063 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d82ae6-f38a-405b-b6dc-764eb627804a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:45.992728 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d82ae6-f38a-405b-b6dc-764eb627804a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.003411 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1356/2733] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:46.007692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.018114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4af7c-ee87-411a-ae04-fa4d5229e429 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.022696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b89c58-6311-42bc-a1d9-ccbaef06ecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.023577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b89c58-6311-42bc-a1d9-ccbaef06ecf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.023858 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b89c58-6311-42bc-a1d9-ccbaef06ecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.024173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b89c58-6311-42bc-a1d9-ccbaef06ecf8 admin admin] PATH_INFO: `/v3/projects/ed8735130a784e78b608019f8ec12d9b/users/73e7f5749bd846c2bebfa0e814e5cd48/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.039321 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b89c58-6311-42bc-a1d9-ccbaef06ecf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8735130a784e78b608019f8ec12d9b, user_id=73e7f5749bd846c2bebfa0e814e5cd48, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.042106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b89c58-6311-42bc-a1d9-ccbaef06ecf8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.057187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1378/2734] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:46 2022] PUT /identity/v3/projects/ed8735130a784e78b608019f8ec12d9b/users/73e7f5749bd846c2bebfa0e814e5cd48/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:46.068575 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173be848-2cf6-463e-938b-2fa8ae21de15 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.068870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173be848-2cf6-463e-938b-2fa8ae21de15 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.069164 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173be848-2cf6-463e-938b-2fa8ae21de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.096957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-173be848-2cf6-463e-938b-2fa8ae21de15 None None] MFA Rules not processed for user `73e7f5749bd846c2bebfa0e814e5cd48`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:46.098165 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-173be848-2cf6-463e-938b-2fa8ae21de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:46.163849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1357/2735] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:46.164381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697447a-c5bb-4c26-9dc8-1976407fcc09 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.164661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697447a-c5bb-4c26-9dc8-1976407fcc09 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.164947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697447a-c5bb-4c26-9dc8-1976407fcc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.168283 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173be848-2cf6-463e-938b-2fa8ae21de15 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.185117 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f697447a-c5bb-4c26-9dc8-1976407fcc09 None None] MFA Rules not processed for user `2de768b55f9a495295113de2ce1279e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:46.186376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f697447a-c5bb-4c26-9dc8-1976407fcc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:46.227660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:46.234655 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173be848-2cf6-463e-938b-2fa8ae21de15 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.239888 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.240616 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.240898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.241170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.241870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.243152 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1379/2736] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:46.244779 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.277411 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1358/2737] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:46 2022] POST /identity/v3/projects => generated 409 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:46.284197 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697447a-c5bb-4c26-9dc8-1976407fcc09 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.292399 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697447a-c5bb-4c26-9dc8-1976407fcc09 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.326465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:46.352665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.353351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.353630 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.353900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.354663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.356626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.380770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1380/2738] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:46 2022] POST /identity/v3/users => generated 311 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:46.382258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.392948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8296c1-71af-425f-abe5-d8608acb7928 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.398220 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.398909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.399195 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.399502 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.400390 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.402562 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.414044 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1359/2739] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:46.418127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.428717 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d2b43-4617-41a6-b9f1-cfb884df4023 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.433671 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.434664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.434941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.435212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] PATH_INFO: `/v3/projects/7697fb21f6e74787ab3ff37f49292f15/users/7d6dd18ad8034320a286732b2533a14f/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.450776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7697fb21f6e74787ab3ff37f49292f15, user_id=7d6dd18ad8034320a286732b2533a14f, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.453467 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.470021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1381/2740] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:46 2022] PUT /identity/v3/projects/7697fb21f6e74787ab3ff37f49292f15/users/7d6dd18ad8034320a286732b2533a14f/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:46.474598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.484998 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0139f-4dfd-40d5-885b-e053114bce80 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.545756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.546489 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.546777 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.547108 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.548055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.549697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.563464 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1360/2741] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:46.565020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.565210 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.566788 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:46.576373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ab211-4528-4360-8873-a1e034e7b37e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.581351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8cb26-a4ce-474f-a4b6-2283c3508fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.582358 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8cb26-a4ce-474f-a4b6-2283c3508fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.582747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8cb26-a4ce-474f-a4b6-2283c3508fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.583022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8cb26-a4ce-474f-a4b6-2283c3508fb7 admin admin] PATH_INFO: `/v3/projects/7697fb21f6e74787ab3ff37f49292f15/users/7d6dd18ad8034320a286732b2533a14f/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.585146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8cb26-a4ce-474f-a4b6-2283c3508fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7697fb21f6e74787ab3ff37f49292f15, user_id=7d6dd18ad8034320a286732b2533a14f, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.587787 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8cb26-a4ce-474f-a4b6-2283c3508fb7 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.598693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1382/2742] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:46 2022] PUT /identity/v3/projects/7697fb21f6e74787ab3ff37f49292f15/users/7d6dd18ad8034320a286732b2533a14f/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:46.609378 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b4cbe-e166-4210-8ac5-72bc5f0c3465 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.609739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b4cbe-e166-4210-8ac5-72bc5f0c3465 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.610052 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b4cbe-e166-4210-8ac5-72bc5f0c3465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.627224 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b921d-32fa-413a-8d2a-af150acfddc1 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.636060 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-b37b4cbe-e166-4210-8ac5-72bc5f0c3465 None None] MFA Rules not processed for user `7d6dd18ad8034320a286732b2533a14f`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:46.637254 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b37b4cbe-e166-4210-8ac5-72bc5f0c3465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:46.677559 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.678308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.678597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.678870 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.680799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:46.690926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.692408 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.700647 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1361/2743] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:58:46.706906 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1383/2744] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:46 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:58:46.728460 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:46.730021 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:46.761997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:46.792384 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97707dc8-fb6f-438a-a434-43fb209ece8a service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:46.796018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:46.796675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:46.796962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:46.797233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:46.806377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:46.807937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:46.817620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1362/2745] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:46 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:58:46.994481 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b4cbe-e166-4210-8ac5-72bc5f0c3465 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:47.004770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b4cbe-e166-4210-8ac5-72bc5f0c3465 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:47.008336 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea38c5-d971-49f2-b6c5-23c713fe44e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:47.008979 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea38c5-d971-49f2-b6c5-23c713fe44e5 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:47.009290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea38c5-d971-49f2-b6c5-23c713fe44e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:47.009526 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea38c5-d971-49f2-b6c5-23c713fe44e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:47.069201 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea38c5-d971-49f2-b6c5-23c713fe44e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:47.070649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea38c5-d971-49f2-b6c5-23c713fe44e5 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:47.080140 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1384/2746] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:46 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:55.585418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:55.653426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a108544-7edb-4b50-a857-708b132712cb service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:55.658374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:55.659286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:55.659546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:55.659834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] PATH_INFO: `/v3/users/73e7f5749bd846c2bebfa0e814e5cd48` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:55.661072 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e7f5749bd846c2bebfa0e814e5cd48)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:55.663004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:55.712889 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e7f5749bd846c2bebfa0e814e5cd48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:55.713632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e7f5749bd846c2bebfa0e814e5cd48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:55.725755 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e7f5749bd846c2bebfa0e814e5cd48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:55.734102 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e7f5749bd846c2bebfa0e814e5cd48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:55.734713 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e7f5749bd846c2bebfa0e814e5cd48'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:55.747169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1363/2747] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:55 2022] DELETE /identity/v3/users/73e7f5749bd846c2bebfa0e814e5cd48 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:55.751892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:55.752412 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:55.752516 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:55.770020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:55.771200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:55.779861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:55.832308 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1385/2748] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:55.842499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:55.844030 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:55.898110 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:55.905093 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee0a3e-8284-4280-a868-af1f12c39267 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:55.910034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:55.911050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:55.911539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:55.911884 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:55.921811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:55.923259 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:55.933137 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1364/2749] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:56.222270 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:56.224021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:56.237146 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bbe7bc-a87a-415c-b4e2-e04b700d00cb None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:56.241897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:56.242941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:56.243187 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:56.243457 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] PATH_INFO: `/v3/projects/ed8735130a784e78b608019f8ec12d9b` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:56.244642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8735130a784e78b608019f8ec12d9b)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:56.246573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:56.311863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8735130a784e78b608019f8ec12d9b', 'request_id': 'req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.318883 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8735130a784e78b608019f8ec12d9b', 'request_id': 'req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.330705 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8735130a784e78b608019f8ec12d9b', 'request_id': 'req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.341863 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] The token cache is being invalidate because project ed8735130a784e78b608019f8ec12d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:56.342143 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.342900 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.344581 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1386/2750] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:56 2022] DELETE /identity/v3/projects/ed8735130a784e78b608019f8ec12d9b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:56.348983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:56.358876 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bee400-2356-4cc7-b8c7-7492d6ac77e7 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:56.415698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:56.416635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:56.416912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:56.417186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] PATH_INFO: `/v3/users/7d6dd18ad8034320a286732b2533a14f` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:56.418389 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6dd18ad8034320a286732b2533a14f)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:56.420369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:56.466922 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6dd18ad8034320a286732b2533a14f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.467580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6dd18ad8034320a286732b2533a14f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.479694 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6dd18ad8034320a286732b2533a14f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.488413 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6dd18ad8034320a286732b2533a14f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.489033 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f02f837e-8221-4af4-91c0-189057217f49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6dd18ad8034320a286732b2533a14f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.501652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1365/2751] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:58:56 2022] DELETE /identity/v3/users/7d6dd18ad8034320a286732b2533a14f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:56.781452 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:56.782984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:56.796295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd8e3a-d6dc-4ae8-ab77-9db69c7fc1b6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:56.857909 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:56.858729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:56.859131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:56.859445 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] PATH_INFO: `/v3/projects/7697fb21f6e74787ab3ff37f49292f15` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:56.861126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7697fb21f6e74787ab3ff37f49292f15)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:56.864067 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:56.905932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:56.919282 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7697fb21f6e74787ab3ff37f49292f15', 'request_id': 'req-229d981f-4a2d-4657-a29f-a420fdf875e3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.928331 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7697fb21f6e74787ab3ff37f49292f15', 'request_id': 'req-229d981f-4a2d-4657-a29f-a420fdf875e3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.936645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7697fb21f6e74787ab3ff37f49292f15', 'request_id': 'req-229d981f-4a2d-4657-a29f-a420fdf875e3'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.947476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] The token cache is being invalidate because project 7697fb21f6e74787ab3ff37f49292f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:58:56.947782 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.948406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-229d981f-4a2d-4657-a29f-a420fdf875e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:58:56.949984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1387/2752] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:58:56 2022] DELETE /identity/v3/projects/7697fb21f6e74787ab3ff37f49292f15 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:56.955002 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2505212-a36c-411b-b8ae-cd1e6cbe2012 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:56.955469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2505212-a36c-411b-b8ae-cd1e6cbe2012 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:56.955582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2505212-a36c-411b-b8ae-cd1e6cbe2012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:56.973815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c2505212-a36c-411b-b8ae-cd1e6cbe2012 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:56.975027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c2505212-a36c-411b-b8ae-cd1e6cbe2012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:57.036199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1366/2753] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:57.045560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511affea-7eac-410f-aa58-3f0bc2ab86ad None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.045854 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511affea-7eac-410f-aa58-3f0bc2ab86ad None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.046130 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511affea-7eac-410f-aa58-3f0bc2ab86ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.066297 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-511affea-7eac-410f-aa58-3f0bc2ab86ad None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:57.067523 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-511affea-7eac-410f-aa58-3f0bc2ab86ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:57.077121 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1388/2754] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:57.081567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2505212-a36c-411b-b8ae-cd1e6cbe2012 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:57.092038 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2505212-a36c-411b-b8ae-cd1e6cbe2012 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:57.098304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:57.098963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.099250 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.099530 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.100304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 15:58:57.100588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:57.102234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:57.104735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:57.118319 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1367/2755] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 15:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 15:58:57.121262 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511affea-7eac-410f-aa58-3f0bc2ab86ad None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:57.134756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511affea-7eac-410f-aa58-3f0bc2ab86ad None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:57.143587 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:57.144753 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.145186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.145588 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.146685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:57.149818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:57.185539 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1389/2756] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:57 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:57.188331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:57.198671 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063523f-723e-4805-988d-1dcca391cbbc admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:57.257639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:57.258324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.258608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.258887 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.259593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:57.261566 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:57.287727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1368/2757] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:57 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:57.289062 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:57.299653 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f4154-337b-4eaa-a445-08f07f9a5821 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:57.305043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:57.305702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.305984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.306258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.306964 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:57.308626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:57.319368 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1390/2758] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 15:58:57.323568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:57.334124 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc4ea41-8ebf-4bc6-b1d6-5e425a80ff5e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:57.339187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc158a-b80c-4d02-b2b3-08c88348a5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:57.339850 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc158a-b80c-4d02-b2b3-08c88348a5ea admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.340213 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc158a-b80c-4d02-b2b3-08c88348a5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.340526 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc158a-b80c-4d02-b2b3-08c88348a5ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.341258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc158a-b80c-4d02-b2b3-08c88348a5ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:57.343444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc158a-b80c-4d02-b2b3-08c88348a5ea admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:57.353631 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1369/2759] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:57.357954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:57.368200 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63fc41-a234-45c8-8583-b648f5a47a31 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:57.373228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:57.374093 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.374373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.374664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] PATH_INFO: `/v3/projects/57296657473b4a64941d9c45e6e06047/users/c5ccb05826fb40bcbe8fd427a6ec3c8f/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.389160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57296657473b4a64941d9c45e6e06047, user_id=c5ccb05826fb40bcbe8fd427a6ec3c8f, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:57.391897 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:57.407756 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1391/2760] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:57 2022] PUT /identity/v3/projects/57296657473b4a64941d9c45e6e06047/users/c5ccb05826fb40bcbe8fd427a6ec3c8f/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:57.413352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66c6f-88c7-4310-9bf3-1cc6a1518dfa None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.413642 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66c6f-88c7-4310-9bf3-1cc6a1518dfa None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.413906 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d66c6f-88c7-4310-9bf3-1cc6a1518dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.432715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c4d66c6f-88c7-4310-9bf3-1cc6a1518dfa None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:57.433923 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c4d66c6f-88c7-4310-9bf3-1cc6a1518dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:57.495158 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1370/2761] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:57.505684 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:57.508702 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:57.570025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a940c-3ead-4b98-9e9b-42b3fd28171c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:57.576519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:57.577250 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:57.577532 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:57.577799 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:57.588428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:57.589856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:57.600184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1392/2762] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:58:58.205832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3d029-daa7-49fa-a7c3-f714abb36b1c None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:58.206363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3d029-daa7-49fa-a7c3-f714abb36b1c None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:58.206498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3d029-daa7-49fa-a7c3-f714abb36b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:58.235269 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0cf3d029-daa7-49fa-a7c3-f714abb36b1c None None] MFA Rules not processed for user `c5ccb05826fb40bcbe8fd427a6ec3c8f`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:58.236549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0cf3d029-daa7-49fa-a7c3-f714abb36b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:58.305132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1371/2763] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:58.309916 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:58.320446 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a696b3d-f39b-4790-a824-966eeb3a7720 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:58.325141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:58.325790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:58.326064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:58.326334 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:58.327107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:58.330956 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:58.356087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1393/2764] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 15:58:58 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:58.360099 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3d029-daa7-49fa-a7c3-f714abb36b1c None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:58.367846 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:58.372425 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3d029-daa7-49fa-a7c3-f714abb36b1c None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:58.430078 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:58.430762 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:58.431042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:58.431339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:58.432064 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:58.433989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:58.455823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1372/2765] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 15:58:58 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 15:58:58.459803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:58.471157 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13f6bb-6187-463f-bcb9-226038883051 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:58.476873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:58.477570 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:58.477853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:58.478125 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:58.478859 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:58.480503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:58.491459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1394/2766] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:58.495644 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:58.505905 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c982f-84a4-4d2f-b5e7-52abb97e4dfc admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:58.510963 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:58.511847 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:58.512152 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:58.512421 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] PATH_INFO: `/v3/projects/e39de64fc952496dbb3ea810ff817691/users/4b54a9882afb40e58ef05cebf07c3c38/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:58.530698 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39de64fc952496dbb3ea810ff817691, user_id=4b54a9882afb40e58ef05cebf07c3c38, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:58.533463 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:58.548709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1373/2767] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:58 2022] PUT /identity/v3/projects/e39de64fc952496dbb3ea810ff817691/users/4b54a9882afb40e58ef05cebf07c3c38/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 15:58:58.553352 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:58.566424 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506359f9-e848-4cb2-9eeb-9b8f2b17f320 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:58.599265 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:58:58.636957 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ce132-e31c-4f20-bd51-e5dfe6e5c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:58.637645 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce132-e31c-4f20-bd51-e5dfe6e5c1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:58.637928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce132-e31c-4f20-bd51-e5dfe6e5c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:58.638203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ce132-e31c-4f20-bd51-e5dfe6e5c1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:58.638940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce132-e31c-4f20-bd51-e5dfe6e5c1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:58.640731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ce132-e31c-4f20-bd51-e5dfe6e5c1e0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:58.650748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1395/2768] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 15:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 15:58:58.655139 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:58.666088 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875304f-d8a1-4637-9797-59c350d693dd admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:58.672024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:58.672937 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:58.673218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:58.673492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] PATH_INFO: `/v3/projects/e39de64fc952496dbb3ea810ff817691/users/4b54a9882afb40e58ef05cebf07c3c38/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:58.675715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39de64fc952496dbb3ea810ff817691, user_id=4b54a9882afb40e58ef05cebf07c3c38, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:58.678446 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:58.689201 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1374/2769] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 15:58:58 2022] PUT /identity/v3/projects/e39de64fc952496dbb3ea810ff817691/users/4b54a9882afb40e58ef05cebf07c3c38/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:58:59.418811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:59.419379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:59.419379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:59.447841 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] MFA Rules not processed for user `4b54a9882afb40e58ef05cebf07c3c38`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:58:59.449097 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:59.522596 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1396/2770] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:58:59.534216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:58:59.536131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:58:59.602467 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a037-95b5-4bea-9fe4-61be052f2842 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:58:59.606377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:58:59.607040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:58:59.607324 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:58:59.607595 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:58:59.668607 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:58:59.669981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:58:59.679954 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1375/2771] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:58:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:59:01.103576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:01.105415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:01.119843 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b67fa-c3f8-4a09-9620-b4863cea4701 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:01.124347 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:01.124984 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:01.125260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:01.125527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:01.136578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:01.138010 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:01.147982 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1397/2772] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:59:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:59:06.651362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:06.653284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:06.669055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:06.719306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15570dd2-12ae-4cc5-b702-f96e9feffd06 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:06.723600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:06.724365 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:06.724653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:06.724931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:06.726742 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:06.799565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:06.801070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:06.811600 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1376/2773] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:59:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:59:11.143578 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=54 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:13.695729 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:13.697401 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:13.772186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb7235-2c4a-43aa-8514-8d5bbd26b879 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:13.778233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:13.778894 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:13.779181 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:13.779441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:13.781492 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:13.794144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:13.795913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:13.806748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1398/2774] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:59:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:59:14.294516 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:14.360675 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8efbc-45bd-4e7a-9fc0-9e8c8ec5fab5 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:14.365780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:14.366657 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:14.366932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:14.367203 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] PATH_INFO: `/v3/users/c5ccb05826fb40bcbe8fd427a6ec3c8f` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:14.368545 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5ccb05826fb40bcbe8fd427a6ec3c8f)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:14.370492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:14.422079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5ccb05826fb40bcbe8fd427a6ec3c8f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.422814 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5ccb05826fb40bcbe8fd427a6ec3c8f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.434824 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:14.436636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5ccb05826fb40bcbe8fd427a6ec3c8f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.446234 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5ccb05826fb40bcbe8fd427a6ec3c8f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.446869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5ccb05826fb40bcbe8fd427a6ec3c8f'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.459378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1377/2775] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:59:14 2022] DELETE /identity/v3/users/c5ccb05826fb40bcbe8fd427a6ec3c8f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:59:14.651503 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:14.663970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e6c186-d74d-4d12-b742-12bdeb58e92d service placement] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:14.725217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:14.725976 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:14.726253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:14.726520 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] PATH_INFO: `/v3/projects/57296657473b4a64941d9c45e6e06047` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:14.727693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57296657473b4a64941d9c45e6e06047)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:14.729792 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:14.778007 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57296657473b4a64941d9c45e6e06047', 'request_id': 'req-032cd16d-b0a9-4650-8fc8-95202d0e3583'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.784194 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57296657473b4a64941d9c45e6e06047', 'request_id': 'req-032cd16d-b0a9-4650-8fc8-95202d0e3583'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.794406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57296657473b4a64941d9c45e6e06047', 'request_id': 'req-032cd16d-b0a9-4650-8fc8-95202d0e3583'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.805536 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] The token cache is being invalidate because project 57296657473b4a64941d9c45e6e06047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:59:14.805819 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.806474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.808072 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1399/2776] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:59:14 2022] DELETE /identity/v3/projects/57296657473b4a64941d9c45e6e06047 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:59:14.812150 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:14.813594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:14.826859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b7895-c0ef-4316-95d7-4e6847b8dbf7 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:14.888715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:14.889601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:14.889883 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:14.890181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] PATH_INFO: `/v3/users/4b54a9882afb40e58ef05cebf07c3c38` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:14.891495 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b54a9882afb40e58ef05cebf07c3c38)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:14.893436 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:14.942155 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b54a9882afb40e58ef05cebf07c3c38'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.942792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b54a9882afb40e58ef05cebf07c3c38'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.954376 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b54a9882afb40e58ef05cebf07c3c38'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.963688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b54a9882afb40e58ef05cebf07c3c38'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.964399 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b54a9882afb40e58ef05cebf07c3c38'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:14.978128 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1378/2777] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 15:59:14 2022] DELETE /identity/v3/users/4b54a9882afb40e58ef05cebf07c3c38 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:59:15.168228 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:15.169778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:15.185797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cd16d-b0a9-4650-8fc8-95202d0e3583 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:15.203829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:15.248627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:15.249415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:15.249696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:15.249970 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] PATH_INFO: `/v3/projects/e39de64fc952496dbb3ea810ff817691` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:15.251209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39de64fc952496dbb3ea810ff817691)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:15.253183 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:15.312749 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39de64fc952496dbb3ea810ff817691', 'request_id': 'req-7c59da03-ac34-4a5c-b213-eadff34ba6a6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:15.320307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39de64fc952496dbb3ea810ff817691', 'request_id': 'req-7c59da03-ac34-4a5c-b213-eadff34ba6a6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:15.329907 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39de64fc952496dbb3ea810ff817691', 'request_id': 'req-7c59da03-ac34-4a5c-b213-eadff34ba6a6'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:15.340410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] The token cache is being invalidate because project e39de64fc952496dbb3ea810ff817691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 15:59:15.340687 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:15.341300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-7c59da03-ac34-4a5c-b213-eadff34ba6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 15:59:15.342875 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1400/2778] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 15:59:15 2022] DELETE /identity/v3/projects/e39de64fc952496dbb3ea810ff817691 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 15:59:24.974620 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=60,keystone:INSERT=2,keystone:DELETE=11 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:25.337363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:37.276570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:37.282672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:37.345691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a67e6-6a93-466c-aeac-46981b070150 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:37.400973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a38e6a-2833-43df-a043-38cdd8ce0c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:37.402115 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38e6a-2833-43df-a043-38cdd8ce0c70 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:37.402462 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38e6a-2833-43df-a043-38cdd8ce0c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:37.402823 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38e6a-2833-43df-a043-38cdd8ce0c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:37.404912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-09a38e6a-2833-43df-a043-38cdd8ce0c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:37.414696 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a38e6a-2833-43df-a043-38cdd8ce0c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:37.416310 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a38e6a-2833-43df-a043-38cdd8ce0c70 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:37.424457 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1379/2779] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:59:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 15:59:37.520750 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa5bb0-dd26-45be-92f7-6b140d64b65a None None] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:37.521483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa5bb0-dd26-45be-92f7-6b140d64b65a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:37.521483 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa5bb0-dd26-45be-92f7-6b140d64b65a None None] PATH_INFO: `/` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:37.523120 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1401/2780] 10.222.0.50 () {54 vars in 921 bytes} [Thu Jun 30 15:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 15:59:37.528378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cae19-6a14-401c-9cae-3072e9a9ee47 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:37.528758 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cae19-6a14-401c-9cae-3072e9a9ee47 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:37.529100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cae19-6a14-401c-9cae-3072e9a9ee47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:37.547982 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-169cae19-6a14-401c-9cae-3072e9a9ee47 None None] MFA Rules not processed for user `4da1612f1c1740ceb2a918f0a8dd057b`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:59:37.549243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-169cae19-6a14-401c-9cae-3072e9a9ee47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:37.620043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1380/2781] 10.222.0.50 () {58 vars in 1018 bytes} [Thu Jun 30 15:59:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 15:59:37.631043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa5bb0-dd26-45be-92f7-6b140d64b65a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 15:59:37.632712 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-65aa5bb0-dd26-45be-92f7-6b140d64b65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:37.647882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa5bb0-dd26-45be-92f7-6b140d64b65a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 15:59:37.652330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 15:59:37.652978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:37.653255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:37.653640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:37.665614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 15:59:37.667476 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 15:59:37.679249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1402/2782] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 15:59:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 15:59:38.134842 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 15:59:38.135248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 15:59:38.135567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 15:59:38.155098 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] MFA Rules not processed for user `9ff354a709914b99bee9785b9577781d`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 15:59:38.156339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 15:59:38.203608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=100 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:38.215060 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1381/2783] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 15:59:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 15:59:47.674474 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=6 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 15:59:48.209187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:00:31.106268 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:00:31.108698 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:00:31.128791 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742531a7-46d7-41f6-93d4-77b2b95c27a7 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:00:31.133035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a967dc1-4fe9-4729-ab22-67e2ff350806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:00:31.133728 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a967dc1-4fe9-4729-ab22-67e2ff350806 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:00:31.133995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a967dc1-4fe9-4729-ab22-67e2ff350806 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:00:31.134271 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a967dc1-4fe9-4729-ab22-67e2ff350806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:00:31.136133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-2a967dc1-4fe9-4729-ab22-67e2ff350806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:00:31.160045 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a967dc1-4fe9-4729-ab22-67e2ff350806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:00:31.161531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a967dc1-4fe9-4729-ab22-67e2ff350806 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:00:31.174114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1403/2784] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:00:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:00:33.039509 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:00:33.041054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:00:33.100034 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df38c14-0986-458a-92ce-5df122392f01 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:00:33.105912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:00:33.106830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:00:33.107186 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:00:33.107536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:00:33.110528 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:00:33.130156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:00:33.131598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:00:33.142772 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1382/2785] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:00:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:00:40.074660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:00:40.075155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:00:40.075276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:00:40.093447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] MFA Rules not processed for user `9ff354a709914b99bee9785b9577781d`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:00:40.094638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:00:40.101699 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1404/2786] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:00:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:00:43.137664 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=26 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:00:50.087746 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=9 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:00.248580 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:00.258610 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0933cc-7999-4ee2-a92e-eb3e52d7bfbd service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:00.262834 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:00.263475 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:00.263757 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:00.264054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:00.266100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:00.332153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:00.333637 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:00.343844 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1383/2787] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:01:06.815534 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:06.817114 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:06.830971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdff119-1a1d-49ee-9452-72e9b510a964 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:06.835557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:06.836465 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:06.836739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:06.837002 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] PATH_INFO: `/v3/users/2de768b55f9a495295113de2ce1279e5` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:06.838192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de768b55f9a495295113de2ce1279e5)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:06.840173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:06.894091 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de768b55f9a495295113de2ce1279e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:06.905954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de768b55f9a495295113de2ce1279e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:06.915003 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de768b55f9a495295113de2ce1279e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:06.915647 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de768b55f9a495295113de2ce1279e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:06.916286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de768b55f9a495295113de2ce1279e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:06.932990 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1405/2788] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:01:06 2022] DELETE /identity/v3/users/2de768b55f9a495295113de2ce1279e5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:01:07.116218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:07.128947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16882468-b4ca-4e17-a282-24e246933336 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:07.189057 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:07.189936 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:07.190217 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:07.190493 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] PATH_INFO: `/v3/projects/affc1e688df04803bb823ccadf904c0a` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:07.191849 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] RBAC: Authorizing `identity:delete_project(project_id=affc1e688df04803bb823ccadf904c0a)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:07.193935 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:07.243930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'affc1e688df04803bb823ccadf904c0a', 'request_id': 'req-e1740b03-0d9e-4f43-8c87-7a122093840b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.253745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'affc1e688df04803bb823ccadf904c0a', 'request_id': 'req-e1740b03-0d9e-4f43-8c87-7a122093840b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.260727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'affc1e688df04803bb823ccadf904c0a', 'request_id': 'req-e1740b03-0d9e-4f43-8c87-7a122093840b'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.273639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] The token cache is being invalidate because project affc1e688df04803bb823ccadf904c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:01:07.273933 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.274969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.276668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1384/2789] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:01:07 2022] DELETE /identity/v3/projects/affc1e688df04803bb823ccadf904c0a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:01:07.280760 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:07.282256 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:07.296495 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fddf2a-a56d-4607-9b59-2b0635cb7bc1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:07.360636 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:07.361537 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:07.361812 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:07.362081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] PATH_INFO: `/v3/users/301f0098c8ce48f2914d5e7ec2a4bc5a` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:07.363291 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=301f0098c8ce48f2914d5e7ec2a4bc5a)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:07.365379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:07.413472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301f0098c8ce48f2914d5e7ec2a4bc5a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.423027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301f0098c8ce48f2914d5e7ec2a4bc5a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.430661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301f0098c8ce48f2914d5e7ec2a4bc5a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.431339 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301f0098c8ce48f2914d5e7ec2a4bc5a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.431936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301f0098c8ce48f2914d5e7ec2a4bc5a'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.440290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:07.453099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1406/2790] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:01:07 2022] DELETE /identity/v3/users/301f0098c8ce48f2914d5e7ec2a4bc5a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:01:07.636154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:07.637635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:07.650761 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1740b03-0d9e-4f43-8c87-7a122093840b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:07.681291 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:07.728568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:07.729496 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:07.729782 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:07.730050 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] PATH_INFO: `/v3/projects/98deee3253214389826d8633dfb172cc` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:07.731308 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98deee3253214389826d8633dfb172cc)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:07.733337 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:07.787722 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98deee3253214389826d8633dfb172cc', 'request_id': 'req-4971d6e1-715d-4dae-bc62-53eeb40faec1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.800318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98deee3253214389826d8633dfb172cc', 'request_id': 'req-4971d6e1-715d-4dae-bc62-53eeb40faec1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.808815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98deee3253214389826d8633dfb172cc', 'request_id': 'req-4971d6e1-715d-4dae-bc62-53eeb40faec1'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.819737 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] The token cache is being invalidate because project 98deee3253214389826d8633dfb172cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:01:07.820051 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.820719 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:07.822401 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1385/2791] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:01:07 2022] DELETE /identity/v3/projects/98deee3253214389826d8633dfb172cc => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:01:08.075673 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:08.077248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:08.185107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fcff9-7fdf-4eb2-b954-cf3b10834cde admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:08.242590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:08.243431 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:08.244333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:08.244333 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:08.246260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:08.258768 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:08.260274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:08.274827 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1407/2792] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:01:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:01:16.462801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:16.464373 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:16.477948 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971d6e1-715d-4dae-bc62-53eeb40faec1 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:16.481896 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:16.483136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:16.483136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:16.483136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:16.485055 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:16.560632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:16.562103 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:16.572304 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1386/2793] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:01:18.275088 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=49,keystone:DELETE=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:18.365866 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:18.367508 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:18.382759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2476-2e50-4a8a-832c-9f81e176d8ce service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:18.388184 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:18.389173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:18.389564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:18.389931 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] PATH_INFO: `/v3/users/6e1730c8601c4334913b13e85e76e8d1` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:18.391310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e1730c8601c4334913b13e85e76e8d1)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:18.393462 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:18.451803 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e1730c8601c4334913b13e85e76e8d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.460470 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e1730c8601c4334913b13e85e76e8d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.467954 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e1730c8601c4334913b13e85e76e8d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.468676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e1730c8601c4334913b13e85e76e8d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.469318 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e1730c8601c4334913b13e85e76e8d1'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.487715 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1408/2794] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:01:18 2022] DELETE /identity/v3/users/6e1730c8601c4334913b13e85e76e8d1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:01:18.678570 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:18.688997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e473d-6d4a-43f7-8c90-2e594d99193f service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:18.742880 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:18.756307 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:18.757187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:18.757504 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:18.757780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] PATH_INFO: `/v3/projects/7b94cd4341d64c409783fecfa81763e1` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:18.758990 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b94cd4341d64c409783fecfa81763e1)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:18.760962 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:18.808362 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b94cd4341d64c409783fecfa81763e1', 'request_id': 'req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.816961 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b94cd4341d64c409783fecfa81763e1', 'request_id': 'req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.823367 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b94cd4341d64c409783fecfa81763e1', 'request_id': 'req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.833693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] The token cache is being invalidate because project 7b94cd4341d64c409783fecfa81763e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:01:18.833981 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.834716 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.836417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1387/2795] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:01:18 2022] DELETE /identity/v3/projects/7b94cd4341d64c409783fecfa81763e1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:01:18.840716 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:18.842243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:18.856305 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cf2a0-60fe-4e2b-884f-80b0aca7a22d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:18.914310 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:18.915199 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:18.915482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:18.915740 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] PATH_INFO: `/v3/users/0036d84503384235b2154dbb33ed9e52` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:18.917987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0036d84503384235b2154dbb33ed9e52)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:18.919977 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:18.971428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0036d84503384235b2154dbb33ed9e52'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.979861 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0036d84503384235b2154dbb33ed9e52'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.986893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0036d84503384235b2154dbb33ed9e52'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.987528 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0036d84503384235b2154dbb33ed9e52'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.988170 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0036d84503384235b2154dbb33ed9e52'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:18.995904 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:19.008258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1409/2796] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:01:18 2022] DELETE /identity/v3/users/0036d84503384235b2154dbb33ed9e52 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:01:19.203169 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:19.204700 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:19.218551 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe3d70-8304-48b4-ae23-89b4f231ef8c admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:19.277286 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:19.278176 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:19.278474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:19.278750 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] PATH_INFO: `/v3/projects/5ca7585f7fca4966950cdce17727c505` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:19.280765 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca7585f7fca4966950cdce17727c505)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:19.282748 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:19.332590 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca7585f7fca4966950cdce17727c505', 'request_id': 'req-25718fca-e79c-4905-adcd-cfc4e1c8f173'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:19.341459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca7585f7fca4966950cdce17727c505', 'request_id': 'req-25718fca-e79c-4905-adcd-cfc4e1c8f173'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:19.348218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca7585f7fca4966950cdce17727c505', 'request_id': 'req-25718fca-e79c-4905-adcd-cfc4e1c8f173'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:19.360202 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] The token cache is being invalidate because project 5ca7585f7fca4966950cdce17727c505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:01:19.360703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:19.361329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-25718fca-e79c-4905-adcd-cfc4e1c8f173 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:01:19.362973 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1388/2797] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:01:19 2022] DELETE /identity/v3/projects/5ca7585f7fca4966950cdce17727c505 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:01:29.007506 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:29.356706 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:01:39.807773 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:39.809711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:39.920731 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd56fb4-f8db-4168-8ba3-142125b1928d admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:40.015718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:40.016987 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:40.017611 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:40.018025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:40.020049 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:40.033857 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:40.035564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:40.047833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1410/2798] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:01:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:01:42.943006 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:42.943554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:42.943554 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:42.962258 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] MFA Rules not processed for user `9ff354a709914b99bee9785b9577781d`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:01:42.963459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:43.021041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1389/2799] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:01:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:01:43.681255 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:43.682725 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:43.754778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f87e0e-bb9d-412a-8b33-36714d553e8f service nova] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:43.761155 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:43.761809 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:43.762094 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:43.762367 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:43.765018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:43.779161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:43.780804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:43.791545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1411/2800] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:01:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:01:50.761303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:01:50.763447 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:50.793695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364c57f-38fb-4ebf-9d0a-98358c284129 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:01:50.800359 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:01:50.801835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:01:50.802132 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:01:50.802391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:01:50.804315 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:01:50.820723 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:01:50.822391 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:01:50.834278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1390/2801] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:01:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:01:53.786459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=94 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:00.829579 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=31 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:47.367126 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:47.368817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:47.439051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce3ec1-b851-45c3-ac2a-af03687a1c82 service placement] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:47.443593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:47.444531 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:47.444806 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:47.445075 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] PATH_INFO: `/v3/users/9ff354a709914b99bee9785b9577781d` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:47.446287 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ff354a709914b99bee9785b9577781d)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:47.448245 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:47.506946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ff354a709914b99bee9785b9577781d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.517568 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ff354a709914b99bee9785b9577781d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.525209 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ff354a709914b99bee9785b9577781d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.525879 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ff354a709914b99bee9785b9577781d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.526557 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-bf40b607-9db7-4162-af8b-493b53771afe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ff354a709914b99bee9785b9577781d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.545313 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1412/2802] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:02:47 2022] DELETE /identity/v3/users/9ff354a709914b99bee9785b9577781d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:47.726349 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:47.738727 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8640c3c-ddb2-4723-9abf-acd4794b6dcd service nova] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:47.794875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:47.795741 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:47.796004 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:47.796306 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] PATH_INFO: `/v3/projects/6c05e9d2132f446a93aba1f27c37b412` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:47.797503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c05e9d2132f446a93aba1f27c37b412)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:47.799431 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:47.851601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c05e9d2132f446a93aba1f27c37b412', 'request_id': 'req-e8eebc3a-5911-476e-9cbf-b21fd53875e9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.860281 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c05e9d2132f446a93aba1f27c37b412', 'request_id': 'req-e8eebc3a-5911-476e-9cbf-b21fd53875e9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.866619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c05e9d2132f446a93aba1f27c37b412', 'request_id': 'req-e8eebc3a-5911-476e-9cbf-b21fd53875e9'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.878334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] The token cache is being invalidate because project 6c05e9d2132f446a93aba1f27c37b412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:02:47.878643 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.879352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-e8eebc3a-5911-476e-9cbf-b21fd53875e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:47.881008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1391/2803] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:02:47 2022] DELETE /identity/v3/projects/6c05e9d2132f446a93aba1f27c37b412 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:47.886429 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7053e-9267-4736-878d-1e58c1a61f6a None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:47.886720 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7053e-9267-4736-878d-1e58c1a61f6a None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:47.886995 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7053e-9267-4736-878d-1e58c1a61f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:47.906826 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-10c7053e-9267-4736-878d-1e58c1a61f6a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:47.908035 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-10c7053e-9267-4736-878d-1e58c1a61f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:47.977573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1413/2804] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:47.986418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8765c0-9639-4002-b71b-2c796831af35 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:47.986688 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8765c0-9639-4002-b71b-2c796831af35 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:47.987003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8765c0-9639-4002-b71b-2c796831af35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.011196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-eb8765c0-9639-4002-b71b-2c796831af35 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:48.012465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-eb8765c0-9639-4002-b71b-2c796831af35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:48.021473 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1392/2805] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:48.025856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7053e-9267-4736-878d-1e58c1a61f6a None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:48.035789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7053e-9267-4736-878d-1e58c1a61f6a None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:48.041341 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:48.042027 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.042307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.042577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.043307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:02:48.043614 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:48.045307 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:48.057606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1414/2806] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 16:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:02:48.061875 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8765c0-9639-4002-b71b-2c796831af35 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:48.071456 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8765c0-9639-4002-b71b-2c796831af35 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:48.076268 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:48.076921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.077207 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.077486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.078185 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:48.080188 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:48.109746 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1393/2807] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:02:48 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:02:48.114292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:48.124416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb47e7-06f2-4862-898e-9492447c1fe0 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:48.128290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:48.186837 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:48.187418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.187700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.187978 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.188707 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:48.190660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:48.211922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1415/2808] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:02:48 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:02:48.216252 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:48.227326 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb56f9e8-a798-4298-86c1-f4667331ee4f admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:48.233695 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:48.234357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.234632 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.234898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.235699 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:48.237340 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:48.248228 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1394/2809] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:02:48.252276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:48.263068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a66520-8026-4d32-bacd-405f7d7238ad admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:48.268933 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b146e92-5fa6-4c8a-a591-d49407e69c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:48.269613 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e92-5fa6-4c8a-a591-d49407e69c5a admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.270292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e92-5fa6-4c8a-a591-d49407e69c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.270292 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b146e92-5fa6-4c8a-a591-d49407e69c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.270873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146e92-5fa6-4c8a-a591-d49407e69c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:48.272560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b146e92-5fa6-4c8a-a591-d49407e69c5a admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:48.283113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1416/2810] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:02:48.287445 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:48.298243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75b988-f00f-4771-9e5b-6f03ade3a2b2 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:48.303574 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:48.304512 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.304794 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.305095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] PATH_INFO: `/v3/projects/1a347e7e2a624c8ba2644d785f9748a7/users/8e904707a1cd4ea9b1fe546e96e5c3e5/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.320791 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a347e7e2a624c8ba2644d785f9748a7, user_id=8e904707a1cd4ea9b1fe546e96e5c3e5, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:48.323459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:48.338733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1395/2811] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:02:48 2022] PUT /identity/v3/projects/1a347e7e2a624c8ba2644d785f9748a7/users/8e904707a1cd4ea9b1fe546e96e5c3e5/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:48.349167 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53ab02-4797-4196-9bdb-9373698b927b None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.349459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53ab02-4797-4196-9bdb-9373698b927b None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.349739 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe53ab02-4797-4196-9bdb-9373698b927b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.376918 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-fe53ab02-4797-4196-9bdb-9373698b927b None None] MFA Rules not processed for user `8e904707a1cd4ea9b1fe546e96e5c3e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:48.378133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fe53ab02-4797-4196-9bdb-9373698b927b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:48.452748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1417/2812] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:48.463317 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:48.465426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:48.497656 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:48.544111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb516-1296-4a8d-a466-5f19decd5343 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:48.550416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a90d7-a82e-4f4a-9a9e-f1428bfd6840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4da1612f1c1740ceb2a918f0a8dd057b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:48.551094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a90d7-a82e-4f4a-9a9e-f1428bfd6840 service nova] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:48.551375 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a90d7-a82e-4f4a-9a9e-f1428bfd6840 service nova] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:48.551652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a90d7-a82e-4f4a-9a9e-f1428bfd6840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:48.564660 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a90d7-a82e-4f4a-9a9e-f1428bfd6840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:48.566302 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a90d7-a82e-4f4a-9a9e-f1428bfd6840 service nova] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:48.575980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1396/2813] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:02:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:02:55.958955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53ab02-4797-4196-9bdb-9373698b927b None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:56.023815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe53ab02-4797-4196-9bdb-9373698b927b None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:56.029533 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:56.030411 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.030686 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.030962 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] PATH_INFO: `/v3/users/8e904707a1cd4ea9b1fe546e96e5c3e5` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.032276 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e904707a1cd4ea9b1fe546e96e5c3e5)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:56.034214 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:56.049901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:56.081851 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e904707a1cd4ea9b1fe546e96e5c3e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.090274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e904707a1cd4ea9b1fe546e96e5c3e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.097564 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e904707a1cd4ea9b1fe546e96e5c3e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.098233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e904707a1cd4ea9b1fe546e96e5c3e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.098892 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e904707a1cd4ea9b1fe546e96e5c3e5'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.116043 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1418/2814] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:02:55 2022] DELETE /identity/v3/users/8e904707a1cd4ea9b1fe546e96e5c3e5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:56.120858 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.121331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.121454 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.140608 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:56.141832 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:56.203230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1397/2815] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:56.213162 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:56.214759 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:56.276415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e98a8-5fa3-4f5f-9a32-9c671d9d27b1 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:56.280521 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debc3a7-8d6b-4861-a582-29d8cbca3a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:56.281173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debc3a7-8d6b-4861-a582-29d8cbca3a5e service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.281458 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debc3a7-8d6b-4861-a582-29d8cbca3a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.281732 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debc3a7-8d6b-4861-a582-29d8cbca3a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.291555 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debc3a7-8d6b-4861-a582-29d8cbca3a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:56.293051 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debc3a7-8d6b-4861-a582-29d8cbca3a5e service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:56.302824 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1419/2816] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:02:56.588119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:56.589582 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:56.605318 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5b412-eff6-4b04-91cf-a1aef7993c75 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:56.611792 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:56.612715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.612983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.613248 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] PATH_INFO: `/v3/projects/1a347e7e2a624c8ba2644d785f9748a7` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.614478 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a347e7e2a624c8ba2644d785f9748a7)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:56.616455 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:56.666070 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a347e7e2a624c8ba2644d785f9748a7', 'request_id': 'req-1d7202f0-3a46-44dc-b54d-da1585fb819c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.678547 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a347e7e2a624c8ba2644d785f9748a7', 'request_id': 'req-1d7202f0-3a46-44dc-b54d-da1585fb819c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.686141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a347e7e2a624c8ba2644d785f9748a7', 'request_id': 'req-1d7202f0-3a46-44dc-b54d-da1585fb819c'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.697009 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] The token cache is being invalidate because project 1a347e7e2a624c8ba2644d785f9748a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:02:56.697294 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.697918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1d7202f0-3a46-44dc-b54d-da1585fb819c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:56.699469 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1398/2817] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:02:56 2022] DELETE /identity/v3/projects/1a347e7e2a624c8ba2644d785f9748a7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:56.704648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a00be-612d-40e3-bb83-c3bec6c27d84 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.704940 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a00be-612d-40e3-bb83-c3bec6c27d84 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.705220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a00be-612d-40e3-bb83-c3bec6c27d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.723887 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-227a00be-612d-40e3-bb83-c3bec6c27d84 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:56.725133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-227a00be-612d-40e3-bb83-c3bec6c27d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:56.788419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1420/2818] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:56.797623 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10622701-8e05-4d93-8611-66b1b6283dd6 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.797909 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10622701-8e05-4d93-8611-66b1b6283dd6 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.798182 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10622701-8e05-4d93-8611-66b1b6283dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.818763 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-10622701-8e05-4d93-8611-66b1b6283dd6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:56.819958 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-10622701-8e05-4d93-8611-66b1b6283dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:56.829653 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1399/2819] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:56.834005 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a00be-612d-40e3-bb83-c3bec6c27d84 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:56.845144 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a00be-612d-40e3-bb83-c3bec6c27d84 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:56.851560 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:56.852248 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.852543 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.852797 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.853522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:02:56.853821 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:56.855499 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:56.868701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1421/2820] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 16:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:02:56.873333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10622701-8e05-4d93-8611-66b1b6283dd6 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:56.883301 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10622701-8e05-4d93-8611-66b1b6283dd6 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:56.888663 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:56.889636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.889917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.890192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:56.890903 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:56.892845 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:56.921993 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1400/2821] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:02:56 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:02:56.926246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:56.936862 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c87bf3-d92b-4f5f-b6e0-40b2f42d3fab admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:56.958231 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:56.998164 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:56.998878 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:56.999105 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:56.999379 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.000106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.002065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.031065 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1422/2822] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:02:56 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:02:57.032357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.043172 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4c681-e927-4441-a506-fe54ecdeb37d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.048101 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.048774 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.049067 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.049328 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.050051 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.051801 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.053231 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:57.067960 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1401/2823] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:02:57.069413 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.079758 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf7b60-c59a-4116-b992-13bc0c3f60e2 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.085762 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f16316-e866-4e38-8f4f-2a7474cbd808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.086419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f16316-e866-4e38-8f4f-2a7474cbd808 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.086690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f16316-e866-4e38-8f4f-2a7474cbd808 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.086960 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f16316-e866-4e38-8f4f-2a7474cbd808 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.087676 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f16316-e866-4e38-8f4f-2a7474cbd808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.089312 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f16316-e866-4e38-8f4f-2a7474cbd808 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.100384 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1423/2824] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:02:57.104571 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.114891 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0c5dc-8ba4-4e80-9acb-beb7d87eb25e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.120532 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.121416 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.121994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.121994 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] PATH_INFO: `/v3/projects/eaf087fcc54e425aa0c9d70be9ecac16/users/0e49846ecb6a4e7595173b9482bf2ecb/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.137279 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf087fcc54e425aa0c9d70be9ecac16, user_id=0e49846ecb6a4e7595173b9482bf2ecb, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.140111 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.157568 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1402/2825] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:02:57 2022] PUT /identity/v3/projects/eaf087fcc54e425aa0c9d70be9ecac16/users/0e49846ecb6a4e7595173b9482bf2ecb/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:02:57.169944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afba12-1fd1-4e41-a4cc-4d0144e70b06 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.170218 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afba12-1fd1-4e41-a4cc-4d0144e70b06 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.170500 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afba12-1fd1-4e41-a4cc-4d0144e70b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.198553 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-30afba12-1fd1-4e41-a4cc-4d0144e70b06 None None] MFA Rules not processed for user `0e49846ecb6a4e7595173b9482bf2ecb`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:57.199946 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-30afba12-1fd1-4e41-a4cc-4d0144e70b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:57.265726 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1424/2826] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:57.270679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.332506 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b7cdf-0633-4562-8df2-7cb63608c8bb admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.338039 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.338724 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.339003 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.339278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.339974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.341915 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.375682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1403/2827] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:02:57 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:02:57.380573 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afba12-1fd1-4e41-a4cc-4d0144e70b06 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.392106 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afba12-1fd1-4e41-a4cc-4d0144e70b06 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.455572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.456290 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.456576 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.456852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.457561 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.459541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.483300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1425/2828] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:02:57 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:02:57.487336 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.497756 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf6642-a757-491a-a0b1-5096aa9edf29 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.502581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.503240 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.503524 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.503802 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.504639 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.506241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.517141 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1404/2829] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:02:57.521240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.531571 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51327e29-bbb1-4a96-a489-b7ad3119934c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.537112 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.537992 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.538273 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.538546 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] PATH_INFO: `/v3/projects/574de495cdd84640bd03fe37993ee64e/users/ea932e85f0cd48e4bdf4c68175cfb076/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.553899 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=574de495cdd84640bd03fe37993ee64e, user_id=ea932e85f0cd48e4bdf4c68175cfb076, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.556575 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.566650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:57.573061 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1426/2830] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:02:57 2022] PUT /identity/v3/projects/574de495cdd84640bd03fe37993ee64e/users/ea932e85f0cd48e4bdf4c68175cfb076/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:57.577020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.587634 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a89f8-d3bd-40a2-88b6-399b0933eac5 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.650548 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ee088-3ddd-4835-a6ba-3c1d020018ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.651331 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ee088-3ddd-4835-a6ba-3c1d020018ef admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.651613 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ee088-3ddd-4835-a6ba-3c1d020018ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.651895 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ee088-3ddd-4835-a6ba-3c1d020018ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.652677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ee088-3ddd-4835-a6ba-3c1d020018ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.654292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ee088-3ddd-4835-a6ba-3c1d020018ef admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.668721 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1405/2831] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:02:57.670081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.680767 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31850d04-0f91-46d3-92a7-7fed5ebf2f43 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.685451 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.686330 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.686604 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.686873 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] PATH_INFO: `/v3/projects/574de495cdd84640bd03fe37993ee64e/users/ea932e85f0cd48e4bdf4c68175cfb076/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.689068 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=574de495cdd84640bd03fe37993ee64e, user_id=ea932e85f0cd48e4bdf4c68175cfb076, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.691810 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.702829 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1427/2832] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:02:57 2022] PUT /identity/v3/projects/574de495cdd84640bd03fe37993ee64e/users/ea932e85f0cd48e4bdf4c68175cfb076/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:57.713053 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6ae0e-68c9-4b39-b598-f42fcf0c9d04 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.713339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6ae0e-68c9-4b39-b598-f42fcf0c9d04 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.713611 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6ae0e-68c9-4b39-b598-f42fcf0c9d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.745181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-0bf6ae0e-68c9-4b39-b598-f42fcf0c9d04 None None] MFA Rules not processed for user `ea932e85f0cd48e4bdf4c68175cfb076`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:57.746873 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-0bf6ae0e-68c9-4b39-b598-f42fcf0c9d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:57.779597 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:57.820237 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1406/2833] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:57.831522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:57.833683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:57.897416 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2d34c-5fed-43e5-a193-56a9c2c2f9ec admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:57.903192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013b4da-def0-4052-9338-f99c32df1ab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:57.903932 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013b4da-def0-4052-9338-f99c32df1ab6 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:57.904227 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013b4da-def0-4052-9338-f99c32df1ab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:57.904527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013b4da-def0-4052-9338-f99c32df1ab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:57.914700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013b4da-def0-4052-9338-f99c32df1ab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:57.916246 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013b4da-def0-4052-9338-f99c32df1ab6 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:57.925627 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1428/2834] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:02:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:02:57.953979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6ae0e-68c9-4b39-b598-f42fcf0c9d04 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:58.020601 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6ae0e-68c9-4b39-b598-f42fcf0c9d04 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:58.025561 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:58.026441 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:58.026720 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:58.026984 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] PATH_INFO: `/v3/users/0e49846ecb6a4e7595173b9482bf2ecb` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:58.028280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e49846ecb6a4e7595173b9482bf2ecb)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:58.030225 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:58.082045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e49846ecb6a4e7595173b9482bf2ecb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.082797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e49846ecb6a4e7595173b9482bf2ecb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.095677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e49846ecb6a4e7595173b9482bf2ecb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.105036 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e49846ecb6a4e7595173b9482bf2ecb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.105790 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e49846ecb6a4e7595173b9482bf2ecb'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.119392 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1407/2835] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:02:57 2022] DELETE /identity/v3/users/0e49846ecb6a4e7595173b9482bf2ecb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:58.124122 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:58.124631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:58.124631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:58.143406 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:58.144642 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:58.204982 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1429/2836] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:02:58.214826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:58.216374 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:58.283500 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:58.289848 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc780c-e322-4153-acf5-efd4ad68df6b admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:58.295752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:58.296629 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:58.297131 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:58.297414 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:58.307594 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:58.309097 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:58.319321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1408/2837] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:02:58.605966 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:58.607489 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:58.620234 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76cd976-bc54-4c1a-9a5e-b6f480662740 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:58.625053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:58.625928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:58.626212 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:58.626478 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] PATH_INFO: `/v3/projects/eaf087fcc54e425aa0c9d70be9ecac16` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:58.627695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf087fcc54e425aa0c9d70be9ecac16)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:58.629683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:58.683690 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf087fcc54e425aa0c9d70be9ecac16', 'request_id': 'req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.690096 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf087fcc54e425aa0c9d70be9ecac16', 'request_id': 'req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.698233 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf087fcc54e425aa0c9d70be9ecac16', 'request_id': 'req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.708595 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] The token cache is being invalidate because project eaf087fcc54e425aa0c9d70be9ecac16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:02:58.708910 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.709530 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.711149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1430/2838] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:02:58 2022] DELETE /identity/v3/projects/eaf087fcc54e425aa0c9d70be9ecac16 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:58.715206 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:58.725396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dafd12-8698-494a-b714-8b79529a8ea8 service neutron] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:58.790697 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:58.791581 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:58.791860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:58.792166 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] PATH_INFO: `/v3/users/ea932e85f0cd48e4bdf4c68175cfb076` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:58.793426 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea932e85f0cd48e4bdf4c68175cfb076)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:58.795371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:58.845534 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea932e85f0cd48e4bdf4c68175cfb076'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.846187 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea932e85f0cd48e4bdf4c68175cfb076'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.858560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea932e85f0cd48e4bdf4c68175cfb076'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.868069 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea932e85f0cd48e4bdf4c68175cfb076'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.868744 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-f0a397fd-2165-4d37-9ba1-ecb42f00ffbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea932e85f0cd48e4bdf4c68175cfb076'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:58.881251 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1409/2839] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:02:58 2022] DELETE /identity/v3/users/ea932e85f0cd48e4bdf4c68175cfb076 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:59.170967 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.173442 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:59.193505 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc5d2a-f9a6-4372-ab57-f5cbce4d8636 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.203754 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:59.256337 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.257131 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.257408 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.257677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] PATH_INFO: `/v3/projects/574de495cdd84640bd03fe37993ee64e` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.258870 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=574de495cdd84640bd03fe37993ee64e)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.260815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:59.309238 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574de495cdd84640bd03fe37993ee64e', 'request_id': 'req-1da37673-c28b-4f31-9cda-670c2ca79ccb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:59.315295 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574de495cdd84640bd03fe37993ee64e', 'request_id': 'req-1da37673-c28b-4f31-9cda-670c2ca79ccb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:59.324944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574de495cdd84640bd03fe37993ee64e', 'request_id': 'req-1da37673-c28b-4f31-9cda-670c2ca79ccb'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:59.335738 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] The token cache is being invalidate because project 574de495cdd84640bd03fe37993ee64e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91957) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:02:59.336018 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:59.336648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-1da37673-c28b-4f31-9cda-670c2ca79ccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:02:59.338204 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1431/2840] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:02:59 2022] DELETE /identity/v3/projects/574de495cdd84640bd03fe37993ee64e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:59.343665 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7ec05-8f93-4333-964f-6eabd1a53e36 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.344560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7ec05-8f93-4333-964f-6eabd1a53e36 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.344560 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7ec05-8f93-4333-964f-6eabd1a53e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.362381 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-b8b7ec05-8f93-4333-964f-6eabd1a53e36 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:59.363587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-b8b7ec05-8f93-4333-964f-6eabd1a53e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:59.423686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1410/2841] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:59.434360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048262dc-f554-478f-91ad-66eae14c6216 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.434667 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048262dc-f554-478f-91ad-66eae14c6216 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.434936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048262dc-f554-478f-91ad-66eae14c6216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.455081 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-048262dc-f554-478f-91ad-66eae14c6216 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:59.456351 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-048262dc-f554-478f-91ad-66eae14c6216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:59.465566 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1432/2842] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:59.470851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7ec05-8f93-4333-964f-6eabd1a53e36 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.486024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7ec05-8f93-4333-964f-6eabd1a53e36 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.492167 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.493090 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.493371 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.493641 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.494377 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91958) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:02:59.494683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.496369 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:59.498838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:59.510917 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1411/2843] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 16:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:02:59.515086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048262dc-f554-478f-91ad-66eae14c6216 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.524838 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048262dc-f554-478f-91ad-66eae14c6216 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.529887 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.530524 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.530818 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.531090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.531784 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.533700 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:59.567535 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1433/2844] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:02:59 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:02:59.568942 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.582868 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68d2ae-5656-4434-839e-407d362e2e99 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.641991 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.642672 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.643024 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.643243 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.644191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.646142 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:59.667314 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1412/2845] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:02:59 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:02:59.671550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.682017 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db448e-f1a9-4137-896d-5234b8cb6474 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.687286 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.687975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.688284 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.688550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.689302 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.691070 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:59.701666 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1434/2846] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:02:59.705879 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.716518 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92f888-3647-4b41-851e-7e52635cf183 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.722154 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e994aa-5406-4449-b522-711e8651dfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.722859 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e994aa-5406-4449-b522-711e8651dfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.723140 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e994aa-5406-4449-b522-711e8651dfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.723419 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e994aa-5406-4449-b522-711e8651dfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.724159 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e994aa-5406-4449-b522-711e8651dfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.725798 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e994aa-5406-4449-b522-711e8651dfb7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:59.735983 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1413/2847] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:02:59.740322 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.750567 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5b584-e9a4-4110-a276-a830092c693c admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.756801 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.757675 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.757951 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.758217 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] PATH_INFO: `/v3/projects/8b6380ce61fe4f199ed8a30d5a98dfcb/users/d1f5a3e8323947348c5c2fed3bf7ec68/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.773586 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6380ce61fe4f199ed8a30d5a98dfcb, user_id=d1f5a3e8323947348c5c2fed3bf7ec68, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.776191 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:02:59.791786 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1435/2848] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:02:59 2022] PUT /identity/v3/projects/8b6380ce61fe4f199ed8a30d5a98dfcb/users/d1f5a3e8323947348c5c2fed3bf7ec68/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:02:59.803338 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebb85-cfd1-4205-9d7f-a37dca483dfe None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.803708 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebb85-cfd1-4205-9d7f-a37dca483dfe None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.804160 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ebb85-cfd1-4205-9d7f-a37dca483dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.837448 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-1a5ebb85-cfd1-4205-9d7f-a37dca483dfe None None] MFA Rules not processed for user `d1f5a3e8323947348c5c2fed3bf7ec68`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:02:59.838648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1a5ebb85-cfd1-4205-9d7f-a37dca483dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:02:59.911589 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1414/2849] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:02:59.916383 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:02:59.959802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:02:59.983343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322533c-ee12-4edd-aab1-cfa4112ca7ed admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:02:59.988563 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:02:59.989192 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:02:59.989472 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:02:59.989748 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:02:59.990512 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:02:59.992459 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.021709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1436/2850] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:02:59 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:03:00.026130 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebb85-cfd1-4205-9d7f-a37dca483dfe None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.037282 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ebb85-cfd1-4205-9d7f-a37dca483dfe None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.098474 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.099149 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.099417 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.099707 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.100440 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.102358 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.111348 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:00.128835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1415/2851] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:03:00 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:03:00.130069 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.140541 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6b0a2-71ff-448e-8de4-ed0e303fb2b2 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.145937 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.146628 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.146912 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.147186 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.147975 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.149606 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.160020 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1437/2852] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:03:00.163918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.174299 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7d64-ca15-47c5-a38f-ff97ea65f32a admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.179245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668a99f-4b3d-4616-ba24-bae614f2b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.179900 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a99f-4b3d-4616-ba24-bae614f2b7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.180212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a99f-4b3d-4616-ba24-bae614f2b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.180486 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668a99f-4b3d-4616-ba24-bae614f2b7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.181222 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668a99f-4b3d-4616-ba24-bae614f2b7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.182815 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668a99f-4b3d-4616-ba24-bae614f2b7d9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.193145 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1416/2853] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:03:00.197441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.208577 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c99497-b61c-4480-b897-e08f7e1fbb06 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.214220 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.215139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] REQUEST_METHOD: `PUT` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.215418 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.215695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] PATH_INFO: `/v3/projects/9d27a54238364b2895ae4a2943ffa6a0/users/88903df69bc84306a03188c44361da46/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.231419 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d27a54238364b2895ae4a2943ffa6a0, user_id=88903df69bc84306a03188c44361da46, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.234087 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.250635 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1438/2854] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:03:00 2022] PUT /identity/v3/projects/9d27a54238364b2895ae4a2943ffa6a0/users/88903df69bc84306a03188c44361da46/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:00.265378 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbabef1-cc58-4302-9bd6-fb8bc0ceebee None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.265666 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbabef1-cc58-4302-9bd6-fb8bc0ceebee None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.265932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbabef1-cc58-4302-9bd6-fb8bc0ceebee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.295735 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-6bbabef1-cc58-4302-9bd6-fb8bc0ceebee None None] MFA Rules not processed for user `88903df69bc84306a03188c44361da46`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:00.296943 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-6bbabef1-cc58-4302-9bd6-fb8bc0ceebee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:00.365083 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1417/2855] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:00.370138 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.444891 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379755f-5a48-44d6-b19c-997b97de408e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.450719 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.451376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.451661 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.451928 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.452648 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.454602 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.484343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1439/2856] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:03:00 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:03:00.488565 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbabef1-cc58-4302-9bd6-fb8bc0ceebee None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.503221 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbabef1-cc58-4302-9bd6-fb8bc0ceebee None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.564947 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.565553 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.565836 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.566113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] PATH_INFO: `/v3/users` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.566811 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.568835 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.589339 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1418/2857] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:03:00 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:03:00.593518 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.604646 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea708c3b-328d-4989-8565-d31ba6ea7e79 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.609708 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.610365 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.610644 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.610917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.611683 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.613428 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.624689 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1440/2858] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:03:00.628916 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.639918 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70697744-e562-46cf-82fc-c2e37c49fe40 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.648976 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.650303 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.650692 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.651027 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] PATH_INFO: `/v3/projects/c808cd6e8847439b9e87806e8ed36eca/users/29436a4897b14de79a5bdff6fed848e4/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.671615 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c808cd6e8847439b9e87806e8ed36eca, user_id=29436a4897b14de79a5bdff6fed848e4, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.674400 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.691499 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1419/2859] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:03:00 2022] PUT /identity/v3/projects/c808cd6e8847439b9e87806e8ed36eca/users/29436a4897b14de79a5bdff6fed848e4/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:00.695701 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.706028 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448050c2-58fa-4524-a174-d25429cde9f6 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.764490 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db9c9b-6665-4e18-8c09-0707cb2f9224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.765150 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9c9b-6665-4e18-8c09-0707cb2f9224 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.765427 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9c9b-6665-4e18-8c09-0707cb2f9224 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.765696 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db9c9b-6665-4e18-8c09-0707cb2f9224 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.766410 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db9c9b-6665-4e18-8c09-0707cb2f9224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.768127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db9c9b-6665-4e18-8c09-0707cb2f9224 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.770179 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:00.780021 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1441/2860] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:03:00.784260 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:00.795191 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e8a91-632e-46f6-9a8d-d73377c777d9 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:00.800974 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:00.801784 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.802081 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.802437 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] PATH_INFO: `/v3/projects/c808cd6e8847439b9e87806e8ed36eca/users/29436a4897b14de79a5bdff6fed848e4/roles/5c802c0e130549e2ba423dbef846a513` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.804703 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c808cd6e8847439b9e87806e8ed36eca, user_id=29436a4897b14de79a5bdff6fed848e4, role_id=5c802c0e130549e2ba423dbef846a513)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:00.807345 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:00.818682 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1420/2861] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:03:00 2022] PUT /identity/v3/projects/c808cd6e8847439b9e87806e8ed36eca/users/29436a4897b14de79a5bdff6fed848e4/roles/5c802c0e130549e2ba423dbef846a513 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:00.896274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006851-0977-452e-b69b-5544162ce645 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:00.896663 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006851-0977-452e-b69b-5544162ce645 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:00.896936 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006851-0977-452e-b69b-5544162ce645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:00.935776 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-09006851-0977-452e-b69b-5544162ce645 None None] MFA Rules not processed for user `29436a4897b14de79a5bdff6fed848e4`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:00.937022 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-09006851-0977-452e-b69b-5544162ce645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:01.010154 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1442/2862] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:01.021334 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:01.023045 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:01.054292 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:01.091841 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3375-5c59-45c1-9fce-bf790610ee4d admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:01.096218 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:01.096861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:01.097136 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:01.097405 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:01.106932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:01.108422 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:01.117910 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1421/2863] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:01.683281 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006851-0977-452e-b69b-5544162ce645 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:01.684695 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-09006851-0977-452e-b69b-5544162ce645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:01.697507 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006851-0977-452e-b69b-5544162ce645 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:01.701438 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:01.702110 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:01.702391 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:01.702664 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:01.769820 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:01.771971 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:01.781350 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1443/2864] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:05.862363 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:05.871931 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21772685-cd8a-4cf8-a4fc-dd2daa34e522 service cinder] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:05.876312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce8869-6828-4ed9-95dd-e067d185486b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:05.876956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce8869-6828-4ed9-95dd-e067d185486b service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:05.877236 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce8869-6828-4ed9-95dd-e067d185486b service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:05.877503 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce8869-6828-4ed9-95dd-e067d185486b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:05.939924 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce8869-6828-4ed9-95dd-e067d185486b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:05.941370 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce8869-6828-4ed9-95dd-e067d185486b service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:05.950715 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1422/2865] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:08.112941 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:08.187161 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b079b28-d182-44db-a141-0e3b5ad29f32 service cinder] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:08.191938 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:08.192845 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:08.193127 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:08.193400 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] PATH_INFO: `/v3/users/d1f5a3e8323947348c5c2fed3bf7ec68` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:08.194618 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1f5a3e8323947348c5c2fed3bf7ec68)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:08.196617 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:08.219913 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:08.246240 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1f5a3e8323947348c5c2fed3bf7ec68'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.255360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1f5a3e8323947348c5c2fed3bf7ec68'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.262693 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1f5a3e8323947348c5c2fed3bf7ec68'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.263390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1f5a3e8323947348c5c2fed3bf7ec68'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.264071 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1f5a3e8323947348c5c2fed3bf7ec68'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.282833 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1444/2866] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:03:08 2022] DELETE /identity/v3/users/d1f5a3e8323947348c5c2fed3bf7ec68 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:08.288032 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:08.288498 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:08.288621 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:08.307095 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:08.308352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:08.380733 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1423/2867] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:08.390590 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:08.392113 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:08.450948 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd88c04-dd1e-40de-83a4-030cf4426ddb admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:08.455083 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:08.455743 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:08.456032 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:08.456324 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:08.467317 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:08.468802 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:08.478848 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1445/2868] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:08.753851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:08.755487 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:08.771380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580ee04-2787-429c-a8a2-1bd97b4adf07 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:08.777497 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:08.778536 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:08.778898 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:08.779087 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] PATH_INFO: `/v3/projects/8b6380ce61fe4f199ed8a30d5a98dfcb` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:08.780538 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6380ce61fe4f199ed8a30d5a98dfcb)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:08.782508 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:08.834585 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6380ce61fe4f199ed8a30d5a98dfcb', 'request_id': 'req-74738806-b12f-4923-a941-4050154ee53e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.843192 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6380ce61fe4f199ed8a30d5a98dfcb', 'request_id': 'req-74738806-b12f-4923-a941-4050154ee53e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.849827 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6380ce61fe4f199ed8a30d5a98dfcb', 'request_id': 'req-74738806-b12f-4923-a941-4050154ee53e'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.851079 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:08.863587 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] The token cache is being invalidate because project 8b6380ce61fe4f199ed8a30d5a98dfcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:03:08.863926 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.864851 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:08.866459 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1424/2869] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:03:08 2022] DELETE /identity/v3/projects/8b6380ce61fe4f199ed8a30d5a98dfcb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:08.870643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:08.880711 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a8feb-8d88-4f7f-8a09-4525cc8b0f74 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:08.944242 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:08.945107 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:08.945381 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:08.945660 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] PATH_INFO: `/v3/users/88903df69bc84306a03188c44361da46` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:08.946921 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88903df69bc84306a03188c44361da46)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:08.948885 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:08.999901 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:09.001847 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88903df69bc84306a03188c44361da46'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.011173 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88903df69bc84306a03188c44361da46'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.018643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88903df69bc84306a03188c44361da46'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.019304 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88903df69bc84306a03188c44361da46'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.019952 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88903df69bc84306a03188c44361da46'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.038372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1446/2870] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:03:08 2022] DELETE /identity/v3/users/88903df69bc84306a03188c44361da46 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:09.338586 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:09.340309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:09.357229 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74738806-b12f-4923-a941-4050154ee53e admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:09.416520 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:09.417396 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:09.417679 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:09.417956 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] PATH_INFO: `/v3/projects/9d27a54238364b2895ae4a2943ffa6a0` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:09.419173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d27a54238364b2895ae4a2943ffa6a0)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:09.421161 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:09.481144 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d27a54238364b2895ae4a2943ffa6a0', 'request_id': 'req-24661303-dd60-4b9f-9423-9cae766ee112'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.492818 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d27a54238364b2895ae4a2943ffa6a0', 'request_id': 'req-24661303-dd60-4b9f-9423-9cae766ee112'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.499465 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d27a54238364b2895ae4a2943ffa6a0', 'request_id': 'req-24661303-dd60-4b9f-9423-9cae766ee112'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.510037 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] The token cache is being invalidate because project 9d27a54238364b2895ae4a2943ffa6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:03:09.510312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.510952 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.512593 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1425/2871] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:03:09 2022] DELETE /identity/v3/projects/9d27a54238364b2895ae4a2943ffa6a0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:09.516692 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:09.518519 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:09.531615 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502e2bf-e4f4-43a1-8f5a-a5bea41cd689 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:09.590626 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:09.591527 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:09.591815 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:09.592136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] PATH_INFO: `/v3/users/29436a4897b14de79a5bdff6fed848e4` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:09.593417 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29436a4897b14de79a5bdff6fed848e4)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:09.595709 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:09.653876 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29436a4897b14de79a5bdff6fed848e4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.662944 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29436a4897b14de79a5bdff6fed848e4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.670139 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29436a4897b14de79a5bdff6fed848e4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.670774 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29436a4897b14de79a5bdff6fed848e4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.671466 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-42e2f083-5c66-4612-a81d-dc86bb24de86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29436a4897b14de79a5bdff6fed848e4'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:09.689817 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1447/2872] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:03:09 2022] DELETE /identity/v3/users/29436a4897b14de79a5bdff6fed848e4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:09.977156 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:09.978691 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:09.992819 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24661303-dd60-4b9f-9423-9cae766ee112 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.054230 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.055043 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.055321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.055598 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] PATH_INFO: `/v3/projects/c808cd6e8847439b9e87806e8ed36eca` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.057005 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c808cd6e8847439b9e87806e8ed36eca)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.058944 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.082799 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:10.111529 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c808cd6e8847439b9e87806e8ed36eca', 'request_id': 'req-891787d6-42cc-44c1-a590-859ffc5db67a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:10.120216 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c808cd6e8847439b9e87806e8ed36eca', 'request_id': 'req-891787d6-42cc-44c1-a590-859ffc5db67a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:10.126352 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c808cd6e8847439b9e87806e8ed36eca', 'request_id': 'req-891787d6-42cc-44c1-a590-859ffc5db67a'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:10.138321 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] The token cache is being invalidate because project c808cd6e8847439b9e87806e8ed36eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:03:10.138606 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:10.139357 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-891787d6-42cc-44c1-a590-859ffc5db67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:10.140997 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1426/2873] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:03:09 2022] DELETE /identity/v3/projects/c808cd6e8847439b9e87806e8ed36eca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:03:10.148222 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aceba1-0290-4a84-a3ba-0c0621ca262c None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.148510 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aceba1-0290-4a84-a3ba-0c0621ca262c None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.148781 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aceba1-0290-4a84-a3ba-0c0621ca262c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.168893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-a7aceba1-0290-4a84-a3ba-0c0621ca262c None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:10.170109 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a7aceba1-0290-4a84-a3ba-0c0621ca262c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:10.209092 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:10.232718 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1448/2874] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:10.241444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255841ff-18c1-4a65-a2c6-4063086c5db6 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.241957 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255841ff-18c1-4a65-a2c6-4063086c5db6 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.242468 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255841ff-18c1-4a65-a2c6-4063086c5db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.263312 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-255841ff-18c1-4a65-a2c6-4063086c5db6 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:10.264683 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-255841ff-18c1-4a65-a2c6-4063086c5db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:10.277830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1427/2875] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:10.282243 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aceba1-0290-4a84-a3ba-0c0621ca262c None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:10.292178 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aceba1-0290-4a84-a3ba-0c0621ca262c None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.296638 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.297300 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.297582 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.297856 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.298591 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:03:10.298923 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.300593 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.312063 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1449/2876] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 16:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:03:10.316424 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255841ff-18c1-4a65-a2c6-4063086c5db6 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:10.326181 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255841ff-18c1-4a65-a2c6-4063086c5db6 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.333128 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.334008 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.334476 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.334908 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.336104 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.338989 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.370617 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1428/2877] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:03:10 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:03:10.375258 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:10.386342 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa8588-c17c-4bd8-9a5d-1960685c3ae8 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.454947 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.455622 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.455868 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.456224 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.456926 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.458871 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.481632 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1450/2878] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:03:10 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:03:10.485843 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:10.497466 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0328ed-e76a-44e6-89d3-c0781f42a133 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.503309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.503972 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.504278 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.504549 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.505276 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.506871 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.518073 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1429/2879] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:03:10.523677 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:10.538415 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3a3b5-57d3-47c0-b08e-f3b6c0fd03dc admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.543920 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13388ce1-e423-4f07-ad5f-d836f8aa72f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.545253 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388ce1-e423-4f07-ad5f-d836f8aa72f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.545538 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388ce1-e423-4f07-ad5f-d836f8aa72f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.545811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13388ce1-e423-4f07-ad5f-d836f8aa72f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.546545 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388ce1-e423-4f07-ad5f-d836f8aa72f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.548160 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13388ce1-e423-4f07-ad5f-d836f8aa72f6 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.559274 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1451/2880] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:03:10.563745 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:10.575054 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844bc3c-c9ec-404c-a98d-d756b5fcfc66 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.580808 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.581739 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.582018 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.582296 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] PATH_INFO: `/v3/projects/ef4fa846b6b543ac9c42a4b87561e683/users/7295653a2f0c4f00ae641238bf91826d/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.598170 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4fa846b6b543ac9c42a4b87561e683, user_id=7295653a2f0c4f00ae641238bf91826d, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.600869 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.616949 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1430/2881] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:03:10 2022] PUT /identity/v3/projects/ef4fa846b6b543ac9c42a4b87561e683/users/7295653a2f0c4f00ae641238bf91826d/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:10.629790 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4da3-c600-4c2b-8cbb-26d731c814ab None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.630077 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4da3-c600-4c2b-8cbb-26d731c814ab None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.630360 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c4da3-c600-4c2b-8cbb-26d731c814ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.656325 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-4e9c4da3-c600-4c2b-8cbb-26d731c814ab None None] MFA Rules not processed for user `7295653a2f0c4f00ae641238bf91826d`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:10.657554 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-4e9c4da3-c600-4c2b-8cbb-26d731c814ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:10.727640 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1452/2882] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:10.740809 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:10.743636 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:10.806901 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce75fb-2547-4982-97c2-d426e786f698 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:10.810710 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df902e73-5432-4729-b305-a854f0b8ad41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4cb4d5213c647459e1c21c311a4a305', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:10.811356 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df902e73-5432-4729-b305-a854f0b8ad41 service glance] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:10.811635 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df902e73-5432-4729-b305-a854f0b8ad41 service glance] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:10.811911 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df902e73-5432-4729-b305-a854f0b8ad41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:10.820860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df902e73-5432-4729-b305-a854f0b8ad41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:10.822233 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df902e73-5432-4729-b305-a854f0b8ad41 service glance] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:10.831300 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1431/2883] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:20.723955 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:20.827578 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:DELETE=7,keystone:SELECT=68,keystone:INSERT=3 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:42.244658 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4da3-c600-4c2b-8cbb-26d731c814ab None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:42.352099 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c4da3-c600-4c2b-8cbb-26d731c814ab None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:42.357206 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:42.358090 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:42.358373 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:42.358650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] PATH_INFO: `/v3/users/7295653a2f0c4f00ae641238bf91826d` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:42.359874 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7295653a2f0c4f00ae641238bf91826d)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:42.361996 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:42.411504 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7295653a2f0c4f00ae641238bf91826d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:42.418779 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7295653a2f0c4f00ae641238bf91826d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:42.425572 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7295653a2f0c4f00ae641238bf91826d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:42.426210 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7295653a2f0c4f00ae641238bf91826d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:42.426816 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7295653a2f0c4f00ae641238bf91826d'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:42.444012 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1453/2884] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:03:42 2022] DELETE /identity/v3/users/7295653a2f0c4f00ae641238bf91826d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:42.449442 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:42.449969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:42.449969 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:42.514020 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:42.515143 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:42.577968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1432/2885] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:42.589294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:42.591030 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:42.635685 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:42.698425 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27916-108e-4611-ae53-1294f35e90dc admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:42.703141 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34b9e1-8a6d-409b-83c6-7c0738e8a1c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:42.703852 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34b9e1-8a6d-409b-83c6-7c0738e8a1c8 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:42.704163 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34b9e1-8a6d-409b-83c6-7c0738e8a1c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:42.704436 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34b9e1-8a6d-409b-83c6-7c0738e8a1c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:42.714363 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34b9e1-8a6d-409b-83c6-7c0738e8a1c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:42.715811 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34b9e1-8a6d-409b-83c6-7c0738e8a1c8 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:42.725800 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1454/2886] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:43.004094 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.005693 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:43.018241 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679ecab-0c28-44ab-8bde-d25159473b6a None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.022780 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.023652 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.023932 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.024238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] PATH_INFO: `/v3/projects/ef4fa846b6b543ac9c42a4b87561e683` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.025418 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4fa846b6b543ac9c42a4b87561e683)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.027329 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.077980 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4fa846b6b543ac9c42a4b87561e683', 'request_id': 'req-968838a4-cdbd-466f-9d2e-cac58af66ca7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:43.087212 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4fa846b6b543ac9c42a4b87561e683', 'request_id': 'req-968838a4-cdbd-466f-9d2e-cac58af66ca7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:43.093479 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4fa846b6b543ac9c42a4b87561e683', 'request_id': 'req-968838a4-cdbd-466f-9d2e-cac58af66ca7'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:43.103921 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] The token cache is being invalidate because project ef4fa846b6b543ac9c42a4b87561e683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:03:43.104238 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:43.104860 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-968838a4-cdbd-466f-9d2e-cac58af66ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:43.106444 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1433/2887] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:03:43 2022] DELETE /identity/v3/projects/ef4fa846b6b543ac9c42a4b87561e683 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:43.111778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400fd1-24ed-4cf5-b5aa-dcb76ce98ba0 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.112133 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400fd1-24ed-4cf5-b5aa-dcb76ce98ba0 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.112482 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09400fd1-24ed-4cf5-b5aa-dcb76ce98ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.158893 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-09400fd1-24ed-4cf5-b5aa-dcb76ce98ba0 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:43.160294 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-09400fd1-24ed-4cf5-b5aa-dcb76ce98ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:43.233420 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1455/2888] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:43.244420 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34639cb9-6c87-4b4f-aafd-83b886264f59 None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.244709 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34639cb9-6c87-4b4f-aafd-83b886264f59 None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.244979 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34639cb9-6c87-4b4f-aafd-83b886264f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.265017 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-34639cb9-6c87-4b4f-aafd-83b886264f59 None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:43.266284 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-34639cb9-6c87-4b4f-aafd-83b886264f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:43.274874 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1434/2889] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:43.279260 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400fd1-24ed-4cf5-b5aa-dcb76ce98ba0 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.289447 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09400fd1-24ed-4cf5-b5aa-dcb76ce98ba0 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.295064 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.295721 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.296031 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.296329 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] PATH_INFO: `/v3/domains` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.297053 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91957) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:03:43.297359 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.299000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.311441 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1456/2890] 10.222.0.50 () {58 vars in 1152 bytes} [Thu Jun 30 16:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:03:43.316380 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34639cb9-6c87-4b4f-aafd-83b886264f59 None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.325912 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34639cb9-6c87-4b4f-aafd-83b886264f59 None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.331592 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.332199 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.332481 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.332752 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] PATH_INFO: `/v3/projects` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.333443 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.335361 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.365280 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1435/2891] 10.222.0.50 () {60 vars in 1152 bytes} [Thu Jun 30 16:03:43 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:03:43.369455 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.380000 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d233c0-57d2-4921-9294-8086dd8a346e admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.436703 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.437372 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.437650 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.437922 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] PATH_INFO: `/v3/users` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.438643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.440665 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.458169 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:43.466616 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1457/2892] 10.222.0.50 () {60 vars in 1143 bytes} [Thu Jun 30 16:03:43 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:03:43.471040 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.481686 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391c5ea-604e-4085-9565-8ec09d0d0389 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.486645 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.487309 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.487588 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.487861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.488624 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.490223 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.500785 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1436/2893] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:03:43.504853 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.515529 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98add2c-0015-40e9-8bc4-0584478f9d09 admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.521609 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ed82e-fd12-4936-a7ea-99fa6c3a8a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.522261 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ed82e-fd12-4936-a7ea-99fa6c3a8a09 admin admin] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.522540 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ed82e-fd12-4936-a7ea-99fa6c3a8a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.522805 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ed82e-fd12-4936-a7ea-99fa6c3a8a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.523522 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ed82e-fd12-4936-a7ea-99fa6c3a8a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.525177 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ed82e-fd12-4936-a7ea-99fa6c3a8a09 admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.535668 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1458/2894] 10.222.0.50 () {58 vars in 1121 bytes} [Thu Jun 30 16:03:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:03:43.539978 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.550546 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffeb98-d915-4003-927b-4ec8fa64ace3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.555711 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.556628 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.556904 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.557173 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] PATH_INFO: `/v3/projects/91ae4fdd407c4774b401b6d63ca87111/users/804a42446f214f81873963b29b15fedf/roles/fd1cdca617c24c089cbe498be4cd05f8` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.576354 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ae4fdd407c4774b401b6d63ca87111, user_id=804a42446f214f81873963b29b15fedf, role_id=fd1cdca617c24c089cbe498be4cd05f8)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.579483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.582830 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:43.604648 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1437/2895] 10.222.0.50 () {60 vars in 1482 bytes} [Thu Jun 30 16:03:43 2022] PUT /identity/v3/projects/91ae4fdd407c4774b401b6d63ca87111/users/804a42446f214f81873963b29b15fedf/roles/fd1cdca617c24c089cbe498be4cd05f8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:43.618153 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944507bf-c80f-4275-a13d-ee2418622d90 None None] REQUEST_METHOD: `POST` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.618448 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944507bf-c80f-4275-a13d-ee2418622d90 None None] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.618724 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944507bf-c80f-4275-a13d-ee2418622d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.647249 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.auth.core [None req-944507bf-c80f-4275-a13d-ee2418622d90 None None] MFA Rules not processed for user `804a42446f214f81873963b29b15fedf`. Rule list: `[]` (Enabled: `True`). {{(pid=91957) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:43.648511 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-944507bf-c80f-4275-a13d-ee2418622d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:43.717136 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1459/2896] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:03:43.728971 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.730996 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:43.789537 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53347-183d-4fc5-8f8c-eca4f067cea3 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.793930 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d9563-c39f-4c77-ba10-d369a2787768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c9cdda3d7884298a791f90523f3ab91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.794567 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9563-c39f-4c77-ba10-d369a2787768 service cinder] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.794838 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9563-c39f-4c77-ba10-d369a2787768 service cinder] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.795112 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9563-c39f-4c77-ba10-d369a2787768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.804190 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9563-c39f-4c77-ba10-d369a2787768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.805619 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9563-c39f-4c77-ba10-d369a2787768 service cinder] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.814541 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1438/2897] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:43.839468 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944507bf-c80f-4275-a13d-ee2418622d90 None None] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:43.912789 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944507bf-c80f-4275-a13d-ee2418622d90 None None] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:43.921280 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:43.922460 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:43.922823 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:43.923190 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] PATH_INFO: `/v3/users/804a42446f214f81873963b29b15fedf` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:43.924985 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=804a42446f214f81873963b29b15fedf)` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:43.926917 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:43.984237 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804a42446f214f81873963b29b15fedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:43.993390 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804a42446f214f81873963b29b15fedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.000456 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804a42446f214f81873963b29b15fedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.001149 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804a42446f214f81873963b29b15fedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.001778 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.notifications [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804a42446f214f81873963b29b15fedf'} {{(pid=91957) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.014770 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:44.020025 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1460/2898] 10.222.0.50 () {58 vars in 1223 bytes} [Thu Jun 30 16:03:43 2022] DELETE /identity/v3/users/804a42446f214f81873963b29b15fedf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:44.024826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] REQUEST_METHOD: `POST` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:44.025113 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:44.025409 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:44.043242 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.auth.core [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] MFA Rules not processed for user `3a26439857af474a895fe8556a7e3453`. Rule list: `[]` (Enabled: `True`). {{(pid=91958) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:03:44.044492 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:44.105539 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1439/2899] 10.222.0.50 () {58 vars in 957 bytes} [Thu Jun 30 16:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:03:44.115714 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:44.117643 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:44.178095 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cba5be-de80-426b-ace2-c4d5a62afe4f admin admin] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:44.181882 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e17d25a83fe48d99ddbd209fbea8486', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:44.182550 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:44.182832 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:44.183134 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:44.192631 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:44.194079 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:44.204086 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1461/2900] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:44.485795 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:44.487677 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:44.504797 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1488560-c6ff-45d9-bf82-939b29cbe3ef None None] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:44.510753 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a26439857af474a895fe8556a7e3453', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c2602e738443cb84400121f999c4f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:44.511659 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:44.512042 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:44.512263 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] PATH_INFO: `/v3/projects/91ae4fdd407c4774b401b6d63ca87111` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:44.513861 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ae4fdd407c4774b401b6d63ca87111)` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:44.515668 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:44.567333 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ae4fdd407c4774b401b6d63ca87111', 'request_id': 'req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.577119 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ae4fdd407c4774b401b6d63ca87111', 'request_id': 'req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.584085 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ae4fdd407c4774b401b6d63ca87111', 'request_id': 'req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918'} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.595196 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] The token cache is being invalidate because project 91ae4fdd407c4774b401b6d63ca87111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91958) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:03:44.595483 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.596153 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.notifications [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91958) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:03:44.597714 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1440/2901] 10.222.0.50 () {58 vars in 1232 bytes} [Thu Jun 30 16:03:44 2022] DELETE /identity/v3/projects/91ae4fdd407c4774b401b6d63ca87111 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:03:48.902176 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] Authenticating user token {{(pid=91957) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:03:48.904422 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:49.013443 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1000734-90a4-4a5a-898c-5308f79f7324 service neutron] Validating token access rules against request {{(pid=91957) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:03:49.076981 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e105e-610c-4043-864c-104463f35649 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91957) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:03:49.077649 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e105e-610c-4043-864c-104463f35649 service placement] REQUEST_METHOD: `GET` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:03:49.077930 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e105e-610c-4043-864c-104463f35649 service placement] SCRIPT_NAME: `/identity` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:03:49.078203 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e105e-610c-4043-864c-104463f35649 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91957) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:03:49.080343 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.fernet_utils [None req-976e105e-610c-4043-864c-104463f35649 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91957) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:03:49.092188 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e105e-610c-4043-864c-104463f35649 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:03:49.093804 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e105e-610c-4043-864c-104463f35649 service placement] RBAC: Authorization granted {{(pid=91957) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:03:49.104747 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: [pid: 91957|app: 0|req: 1462/2902] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:03:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:03:54.591968 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:INSERT=2,keystone:SELECT=82,keystone:DELETE=7 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:03:59.100376 dv-cinder-797608-40-iscsi devstack@keystone.service[91957]: DEBUG dbcounter [-] [91957] Writing DB stats keystone:SELECT=95 {{(pid=91957) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:08:54.944100 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Authenticating user token {{(pid=91958) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:08:54.945751 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.fernet_utils [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91958) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:08:54.962245 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9f2b6-5ff4-42f7-a96c-3cda48e0e918 admin admin] Validating token access rules against request {{(pid=91958) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:08:54.969041 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad04b9ff-c057-495a-bae3-a5aa6af2d79f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e9bc6cff2d9453a80cfce2c7e66d2e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c25a6e43fdc045ada347b484dfdbe8ee', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91958) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:08:54.969687 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04b9ff-c057-495a-bae3-a5aa6af2d79f service placement] REQUEST_METHOD: `GET` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:08:54.969967 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04b9ff-c057-495a-bae3-a5aa6af2d79f service placement] SCRIPT_NAME: `/identity` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:08:54.970239 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad04b9ff-c057-495a-bae3-a5aa6af2d79f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91958) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:08:54.979270 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04b9ff-c057-495a-bae3-a5aa6af2d79f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:08:54.980890 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad04b9ff-c057-495a-bae3-a5aa6af2d79f service placement] RBAC: Authorization granted {{(pid=91958) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:08:54.990807 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: [pid: 91958|app: 0|req: 1441/2903] 10.222.0.50 () {60 vars in 1365 bytes} [Thu Jun 30 16:08:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:09:04.986826 dv-cinder-797608-40-iscsi devstack@keystone.service[91958]: DEBUG dbcounter [-] [91958] Writing DB stats keystone:SELECT=6 {{(pid=91958) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}